r 

AD-A012  477 

NATURAL  LANGUAGE 
A.  I.  PARADIGM: 

UNDERSTANDING  SYSTEMS  WITHIN  THE 
A SURVEY  AND  SOME  COMPARISONS 

Yo'^ick  Wilks 

Stanford  University 

J 

r 


Prepared  for: 

Advanced  Research  Projects  Agency 
Office  of  Naval  Research 


December  1974 


CISTRiBUTED  BY; 


Ndtiona!  Tec^^icai  Information  Service 
U.  S.  DEPARTMENT  OF  COMMERCE 


ADA01247 


^0914? 


Stanford  Artificial  Ir.ielligence  Laboratory  December  1974 

Memo  AIM- 23  7 

Computer  Science  Department 
Report  No.  STAN-CS-74-436 


Natural  Language  Understanding  Systems 
Within  the  AI  Paradigm 

A Survey  and  Some  Comparisons 
by 


Yorick  Wilks 


Research  sponsored  by 

Advanced  Research  Projects  Agency 
ARPA  Order  No.  2494 


COMPUTER  SCIENCE  DEPARTMENT 
Stanford  University 


nationa;  technical 

INFORMATION  SERVICE 


H, 


t 


UNCLASSIFIED 


security  classification  of  this  page  flTA«n  Data  KntaiaJ) 


REPORT  DOCUMENTATION  PAGE 


I.  report  number  li.  govt 

STAH-CS-74-436  I 


12.  GOVT  ACCESSION  NO. 


4.  T\TLE  (and  Subtitte) 

NATURAL  LANGUAGE  UNDERSTANDING  SYSTliMS  WITillN  TfiE 
A. I.  PARADIGM:  A SURVEY  AND  SOME  COMPARISONS. 


7.  AUTHORfAj 


READ  INSTRUCTIONS 
BEFORE  COMPLETING  FORM 


3.  RECIPIENT'S  CAT  ALOG  NUMBER 


s.  TYPE  OF  REPORT  ft  PERIOD  COVERED 

technical,  Dec.  1974 


6.  PERFORMING  CRG.  REPORT  NUMBER 

ITAN-CS- (also  AM  ?37) 


8.  CONTRACT  OR  GRANT  NUMBERf.J 


Yorick  Wilks 


DAHC  15-73-C-0435 


9.  PERFORMING  ORGANIZATION  NAME  AND  ADDRESS 

Computer  Science  Department 
Stanford  University 
Stanford,  California  9^305 


II.  CONTROLLING  OFFICE  NAME  AND  ADDRESS 

ARPA/IFT,  Attn:  S.  D.  Crocker 

l400  Wilson  Blvd.,  Arlington,  Va.  22k09 


14.  MONITORING  AGENCY  NAME  8 ADDRESSfl/  dlllatrnt  Irom  Controlling  Ollica) 

ONR  Representative:  Philip  Surra 

Durand  Aeronautics  Bldg.,  Rm.  l65 
Stanford  University 
Stanford,  California  94305 


15  DISTRIBUTION  STATEMENT  Col  Mi  Roport) 

Releasable  without  limitations  on  dissemination. 


10.  PROGRAM  ELEMENT.  PROJECT,  TASK 
AREA  a WORK  UNIT  NUMBERS 


12.  REPORT  DATE 

Dec.  1974 


13  NUMBER  OF  PAGES 


- 

IS.  SECURI 


URITY  CLASS,  (ol  Ihia  raport) 


Unclassified 


I5».  DECLASSIFICATION  downgrading 
SCHEDULE 


17.  Distribution  statement  (ot  the  abntract  <9nter0d  in  dtock  20,  tt  difterent  from  Report) 


19.  KEY  WORDS  (Continue  on  rover.ne  nice  it  neceesmry  and  ider.tlly  by  block  number) 


20  abstract  (Continue  on  reverae  aide  il  neceamary  end  identlly  by  block  number) 

The  paper  surveys  the  major  projects  on  the  understanding  of  natural  language 
that  fall  within  what  may  now  be  called  the  artificial  intelligence  paradigm 
for  natural  language  systems.  Some  space  is  devoted  to  arguing  that  the 
paradigm  is  now  a reul'ty  and  different  in  significant  respects  from  the 
generative  paradigi'  resent  day  linguistics.  The  comparisons  between 
systems  center  ar-'  .estions  of  the  relative  perspicuity  of  pncedu.ral 

and  static  repre  ..oions;  the  advantages  and  disadvantages  of  developing 
systems  over  a od  Survey  and  Some  Comparisons. 


FORM 
JAN  73 


EDI' ION  OF  I NOV  6S  IS  OBSOLETE 


TO  CHANGE 


security  classification  of  THI:.  page  (When  nata  Entered) 


Stanford  Artificial  Intelligence  Laboratory 
Memo  AIM-237 


December  1974 


Comput-ir  Science  Department 
Report  K'o.  STAN-CS-74-436 


Natural  Language  Unclerstnnding  Systems 
Within  the  AI  Paradigm 

A Survey  and  Some  Comparisons 
by 

Yoriuk  Wilks 


ABSTRACT 


The  pajjer  surveys  the  major  projects  on  the  unacrstanding  of  natural  language  that  fall 
within  what  may  now  be  cailed  the  artificial  intelligence  paradigm  for  natural  language 
systems.  Some  space  is  devoted  to  arguing  that  the  paradigm  is  now  a reality  and  different  in 
significant  respects  from  the  generative  paradigm  of  present  day  linguistics.  The  comparisons 
between  systems  center  around  questions  of  the  relative  perspicuity  of  procedural  and  static 
representations;  the  advantages  and  disadvantages  of  developing  systems  over  a period  Survey 
and  Some  Comparisons 


This  research  was  supported  by  the  Advanced  Research  Projects  Agency  of  tne  Departrnent  of 
Defense  under  Contract  DAHC  li-7TC-04}3  . The  views  and  conclusions  contained  in  this 
document  are  those  of  the  author(s)  and  should  not  be  interpreted  as  necessarily  representing  the 
official  policies,  either  expressed  or  implied,  of  Stanford  University,  ARPA,  or  the  V.  S. 
Government. 

Reproduced  in  the  U S. A,  Available  from  the  National  Technical  Information  Service,  Springfield, 
Virginia  22131 


J/ 


/ 


1.  INTRODUCTION 


In  his  report  to  the  Science  Reiearch  Council  on  the  state  of  Artificial  Intelligence,  Sir  James 
Lighthill  (197?)  gave  most  of  the  field  a rather  ba  ! 'gnosis.  One  of  the  few  hopeful  signs  he 
saw  was  Winograd's  (1972)  natural  language  understanding  system.  Yet  now,  o'lly  a year  later, 
Winograd  has  stopped  work  on  the  system  he  has  r-nstructed,  and  has  begun  a new  one  on 
entirely  different  principles.  He  went  so  far.  in  a survey  lecture  (Winograd  1973)  of  extraordinary 
modesty  in  a field  not  known  for  its  small  claims,  to  place  his  celebrated  early  work  in  only  the 
■first  generation’  of  computer  systems  designed  to  understand  natural  language,  and  went  on  to 
describe  others'  'second  generation'  systems. 

I shall  return  later  to  this  metaphor  of  generations,  but  what  is  one  to  say  in  general  terms  of  a 
field  where  yesterday’s  brightest  spots  are  today’s  first  generation  sy.tems,  even  though  they  have 
not  been  criticised  in  print,  not  shown  in  any  generally  acceptable  way  to  be  fundamentally 
wrong?  Part  of  the  answer  lies  in  the  profound  role  of  fashion  in  Artificial  Intelligence  in  its 
present  pre-scientific  phase  A cynical  American  professor  remarked  recently  that  Artificial 
Intelligence  (Al)  had  an  affair  with  someone’s  wo-k  every  year  or  two.  and  that,  just  a'  there  were 
no  reasons  for  falling  in  love,  so,  later,  there  were  no  reasons  for  falling  out  again.  In  the  case  cf 
Winograd’s  work  it  is  important  now  to  resist  this  fashion,  and  re-emphasize  what  a good  piece  of 
research  it  was.  as  1 shall  in  a moment 


Another  part  of  the  answer  lies  in  the  still  fundamental  role  of  metaphysical  criticism  in  Al.  In 
the  field  of  computer  vision  things  are  bad  enough,  in  that  anybody  who  can  see  feels  entitled  to 
criticise  a system,  on  the  ground  that  he  is  sure  he  does  not  see  using  such  and  such  principles.  In 
the  field  of  natural  language  understanding  things  are  worse:  not  only  does  anyone  who  can 
speak  and  write  feel  free  to  criticise  on  the  conespording  grounds,  but  in  addition  there  are  those 
trained  in  disciplines  parasitic  upon  natural  language,  linguists  and  logicians,  who  often  know  in 
addition  how  things  MUST  BE  DONE  on  a prion  grounds.  It  is  this  metaphysical  aspect  of  the 
subject  that  g-ves  its  disputes  their  characteristically  acrimonious  flavour. 


In  this  paper  I want  to  sort  out  a little  what  is  agieed  and  what  is  not;  what  are  some  of  the 
outstanding  disputes  and  how  testable  are  the  claims  being  made?  If  what  follows  seems  unduly 
philosophical,  it  should  be  remembered  that  little  i;  agreed,  and  almost  no  achievements  are 
beyond  question.  To  pretend  otherwise,  by  concentrating  only  on  the  details  of  established 
programs,  would  be  meretricious  and  misleading 

To  survey  an  energetic  field  like  this  one  is  inevitably  to  leave  a great  deal  of  excellent  work 
unexamined,  at  least  if  one  is  going  to  do  more  than  give  a paragraph  to  each  research  project.  I 
have  left  out  of  consideration  at  least  six  groups  of  projects: 


(1)  Early  work  in  Artificial  Intelligence  and  Natural  Language  that  has  been  surveyed  by 
Winograd  (1973)  and  Simmons  (I970a;  among  others: 

(2)  Work  by  graduate  students  of,  or  intellectually  dependent  upon  that  of,  people  discussed 
in  some  detail  here; 

(3)  Work  that  derives  essentially  fron’  projects  described  in  detail  here.  This  embraces 
several  groups  interested  in  testing  psycholog.cal  hypotheses,  as  well  as  others  constructing 
large-scale  systenis  for  speech  recognition.  I have  devoted  no  space  to  speech  recognition  as 


‘Oini'r 
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such  here,  for  it  seems  to  me  to  depend  upon  the  quality  of  semantic  and  iriferential 
understanding  as  much  as  anything,  and  so  1 have  concentrated  upon  this  more  fundamental 
task; 

(“1)  Work  on  language  generators,  as  opposed  to  analysers  and  understanders.  They  are 
essential  for  obtaining  any  testable  output,  but  are  theortically  secondary; 

(5)  All  the  many  and  varied  reasoning  schemes  now  available  in  Al.  including  PLANNER 
(Hewitt  1969),  QA4  (Rulifson  et  al  1972),  MERLIN  (Moore  and  Newell  1973),  as  well  as 
automatic  programming  (Balzer  et  al  1974)  (Heidorn  1974)  and  debugging  projects  (Sussman 
1974),  many  of  which  are  producing  formalisms  that  appear  increasingly  like  natural  language; 

(6)  Conservative  reasoning  schemes,  such  as  first  order  predicate  calculus,  that  have  been 
applied  to,  or  advocated  for,  the  analysis  of  natural  language;  (McCarthy  and  Hayes  1969) 
(Coles  1972)  (Sandewall  1972) 


The  e.xcIusions  under  (2)  above  are  particularly  unfair  to  the  work  in  the  unpublished  theses  of 
Rieger  (1974)  and  McDermott  (1974),  and  can  only  be  Justified,  like  those  of  sections  (5)  and  (3) 
above,  by  space,  bias  and  the  considerations  advanced  in  a later  section  of  this  survey  concerning 
what  it  is  for  A I research  to  be  about  natural  language,  rather  than  about  something  else,  like 
human  psychology,  or  logic,  or  the  bargaining  behavior  of  children  under  stress. 

One  final  remark  about  the  style  of  this  survey.  1 have  not  indulged  in  the  normal  critical  luxury 
of  careful  and  extensive  quotation  from  authors  followed  by  detailed  and  meticulous  criticism. 
Th.at  would  be  wholly  inappropriate  in  the  present  state  cf  things.  .A  great  deal  of  work  is  being 
done  at  the  moment,  and  many  of  the  principal  researchers  change  their  views  on  very 
fundamental  questions  between  one  paper  and  the  next  without  drawing  any  attention  to  the  fact. 
Cheap  self-contradictions  and  changes  of  mind  are  all  too  easy  to  find,  so  criticism  and 
comparisons  are  best  drawn  with  a very  broad  brush  and  a light  stroke 


2.  WINOCRAD’S  UNDERSTANDING  SYSTEM 


Winograd’s  system  runs  as  a dialogue,  in  real  time,  between  a human  operator  who  types  m 
messages,  and  the  system  proper,  called  SHRDLU,  which  displays  or.  a screen  pictures  of  a closed 
world  of  coloured  blocks  and  pyramids,  a box  into  which  they  can  be  put.  and  an  arm  or  crane 
that  can  move  the  objects  about.  SHRDLU  displays  its  replies  to  the  operator  by  writing  on  tiie 
same  screen.  The  following  is  a discontinuous  sample  of  the  dialogue  Winograd  gives  in  his 
book.  SHRDLU’s  replies  are  in  upper  case: 


pick  up  the  big  red  block 
Ok 

find  a block  which  is  taller  than  the  one  you  are  holding  and  put  it 
into  the  boy. 

BY  ‘IT’  I ASSUME  YOU  MEAN  THE  BLOCK  WHICH  IS  TALLER  THAN  THE  ONE  I AM  HOLDING 
OK 

iihal  cioes  the  box  contain 

THE  BLUE  PYRAMID  AND  THE  BLUE  BLOCK 

had  you  touched  any  pyramid  before  you  put  the  green  one  on  the  little  cube 
YES.  THE  GREEN  ONE 
when  did  you  pick  it  up 

WHILE  I WAS  STACKING  UP  THE  REO  CUBE.  A LARGE  REO  BLOCK  AND  A LARGE  GREEN 

CUBE 

why 

TO  GET  RIO  OF  IT 


This  ingenious  program  was  also  able  to  accept  new  definitions  of  words,  such  as  ‘steeple’;  to  make 
use  of  what  it  knew  about  the  properties  of  the  objects  described,  such  as  that  a block  could  have 
Its  top  cleared  off,  but  a pyramid  could  not;  and  a'so  to  remember  what  it  had  done  before,  as  in 
the  sample  above. 

The  program  was  written  in  the  language  PLANNER,  (I)  which  is  a concrete  expression  of  the 
slogan  "meanings  are  procedures",  a sentiment  into  whose  own  meaning  it  is  probably  best  not  to 
inquire  too  closely,  but  which  has  undoubtedly  led  to  a new  style  of  programming.  PLANNER  is 
a theorem  proving  language:  it  tries  to  establish  the  truth  of  assertions,  not  in  tl.e  normal 
uniform,  proof-theoretic,  manner,  but  by  accepting  a range  of  "programmed  hints"  abou'  how  best 
to  proceed  at  any  point.  In  a language  understanding  program  like  Winograd’s,  this  means 
replacing  familiar  grammar  rules  such  as  S-*NP  ♦ VP  (a  sentence  consists  of  a noun  phrase 
followed  by  a verb  phrase)  by  procedures,  in  this  case: 


((PDEFINE  SENTENCE  ((PARSE  NP)  NIL  FAIL)  ((PARSE  VP)  FAIL  FAIL  RETURN))) 


The  details  of  the  notation  need  not  detain  us;  what  is  important  is  that  Winograd’s  grammar  is 
not  the  conventional  list  of  rules,  but  small  .sub-programs  like  the  lines  above,  that  actually 


(I)  Actually,  in  a sub  set  of  PLANNER  called  PROCRAMMAR.  (Hewitt,  1969) 


•1 


represent  procedures  for  imposing  the  desired  grammatical  structure,  The  definitions  of  more 
complex  words  are  also  in  this  form  here,  for  example,  is  the  ‘theorem’  defining  the  content  of 

‘pickup’: 


(DEFTHEOREM  TC-PICKUP  (THCONSE  (X(WHY  (EV))EV) 

(•PICKUP  8’X)  (MEMORY)  (THGOAL(»GRASP  l?X)  (THUSE  TC-GRASP)) 
(THGOAL  (*RAlSEHANr'  'thNODB)  (THUSE  TC-RAISEHAND)) 
(MEMOREND  (.P’.CKUP  «?EV  «?X)))) 


Once  acain  the  details  of  the  notation  need  not  be  explained  in  order  to  see  that  the  word  is 
belli?  denied  in  terms  of  a number  of  more  primitive  sub-actions,  such  as  RAISEHAND,  each  of 
which  must  be  carried  out  in  order  that  something  may  indeed  be  picked  up.  The  linguistic 
content  is  a vocabulary  of  what  seems  to  be  about  175  words,  a ‘systemic  grammar’,  due  ru  M,  A 
K Halliday  (1970),  plus  a simple  system  of  semantic  ‘features’,  marking  words  and  arranged 
hierarchically,  such  as  PHYSOB  (for  physical  object  words)  and  ANIMATE  (for  ‘animate  words’ 
like  ‘robot’)  together  with  some  factual  knowledge  about  the  block  world.  Both  types  of 
knowledge,  linguistic  and  factual,  are  represented  in  PLANNER  when  the  program  actually  runs, 
and  if  is  able  to  access  whichever  sort  is  required  at  any  given  moment,  rather  than  in  the 
conventional  manner,  of  first  doing  syntactic  parsing  to  get  a syntactic  structure  and  then 
manipulating  the  features  to  get  .a  semantic  structure. 

One  reason  for  the  enormous  impact  of  this  work  was  that,  prior  to  its  appearance,  A I work  was 
linguistically  trivial,  while  the  systems  of  the  linguists  had  no  place  for  the  use  of  inference  and 
real  world  knowledge.  Thus  a very  limited  union  between  the  two  techniques  was  able  to  breed 
considerable  results'  Before  Winograd  there  were  few  programs  in  A!  that  could  take  a 
reasonable  complex  English  sentence  and  ascribe  any  structure  whatever  to  it.  In  early  classics  of 
‘natural  language  understanding’  in  AI.  such  as  Bobrow’s  STUDENT  (1968)  problem  solver  for 
simple  algebra,  input  sentences  had  to  be  short  and  of  stereotyped  form,  such  as  ‘what  is  the  sum 

of...’’ 

Conversely,  in  linguistics,  there  was,  until  very  recently,  little  speculation  on  how  we  understand 
the  reference  of  pronouns  in  such  elementary  sentences  as  "the  soldiers  fired  at  the  women  and  I 
saw  several  fall",  where  it  is  clear  that  the  answer  is  both  definite,  and  that  finding  it  requires 
some  inferential  manipulation  of  generalisations  about  the  world.  The  reader  should  ask  himself 
at  this  point  how  he  Knows  the  referent  of  the  pronoun  in  that  sentence. 


3.  SOME  DISCUSSION  OF  SHRDLU 


So  far  the  reaction  to  Winograd's  work  has  been  wholly  uncritical  What  would  critics  find  tn 
attack  'if  they  were  so  minded?  Firstly,  that  Winograd's  linguistic  system  is  highly  conservative. 
Tnd  that  the  distinction  between  ‘syntax’  and  ’semantics’  nay  not  be  necessary  at  all.  Secondly, 
that  his  semantics  is  tied  to  the  simple  referential  work  of  the  blocks  in  a way  that  would  make  it 
ncxtensible  to  any  general,  real  world,  situation  Suppose  ’block  were  allowed  to  mean  an 
obstruction’  and  ’a  mental  inhibition’,  as  well  as  ’a  cubic  object.  It  is  doubtful  whether 
Winottrad's  features  and  rules  could  express  the  ambiguity,  and.  more  importantly,  whether  the 
simole  structures  he  manipulated  could  decide  correctly  between  the  alternative  meanings  in  any 
given  context  of  use.  Again,  far  more  sophistocated  and  systematic  case  structures  than  those  he 
Led  might  be  needed  to  resolve  the  ambiguity  of  ‘in’  in  ’He  i..n  the  mile  in  five  minutes,  and  He 
ran  the  mile  in  a paper  bag’,  as  well  as  the  combination  of  case  with  word  sense  ambiguity  in  He 
put  the  key  in  the  lock’  (door  lock)  and  ’He  threw  the  key  in  the  lock’  (river  lock). 

The  blocks  world  is  also  strongly  deductive  and  logically  closed.  If  gravity  were  introduced  into  it. 
then  anything  supported  that  was  pushed  in  a certain  way  would  have,  logically,  to  fall._  But  the 
common  sense  worW.  of  ordinary  language,  is  not  like  that:  in  the  women  and  scridiers  example 
given  earlier  the  pronoun  ’several’  can  be  said  to  be  resolved  using  some  generalisation  such  .as 
things  shot  at  and  hurt  tend  to  fall’.  There  are  no  logical  ’have  to’s’  there,  even  though  the 
meaning  of  the  pronoun  is  perfectly  definite 

Indeed  it  might  be  argued  that,  in  a sense,  and  as  regards  its  semantics,  Winogracl’s  system  is  not 
about  natural  language  at  all.  but  about  ih>  other  technical  question  of  how  goals  and  subgoals 
are  to  be  organisL  in  a problem  .solving  system  capable  of  manipulating  simple  physical  objects. 
If  one  glances  back  at  the  definition  of  ’pickup’  quoted  above,  one  can  see  that  it  is  in  fact  an 
expiessfon  of  a procedure  for  picking  up  an  object  in  the  SHRDLU  system  Nothing  about  it  for 
example  would  help  one  understand  the  perfectly  ordinary  sentence  1 picked  up  my  bags  fiom 
the  platform  and  ran  for  the  train’.  One  could  put  the  point  so:  what  we  are  given  in  the 
planner  code  is  not  a sense  of  ’pick  up  but  a case  of  its  use,  just  as  John  picked  up  le 
volunteer  from  the  audience  by  leaning  over  the  edge  of  the  stage  and  drawing  her  up  by  means 
of  a rope  clenched  in  his  teeth’  is  not  so  much  a sense  of  the  verb  as  a use  of  it. 

Those  who  like  very  general  analogies  may  have  noticed  that  Wittgenstein  (1953  para.  2f(.) 
devoted  considerable  space  to  the  construction  of  an  elementary  language  of  blocks,  beams  and 
Slabs  one  postulated  on  the  assumption  that  the  words  of  language  were  basically,  as  is  supposed 
in  model  theory  the  names  of  items.  But  he  showed  of  the  enterprise,  and  to  the  satisfaction  of 
manv  readers  ’’That  the  philosophical  concept  of  meaning  (le  of  words  as  the  unambiguous 
names  of  physical  objects-YW)  has  its  place  in  a primitive  idea  of  the  way  language  functions. 
But  one  can  also  say  that  it  is  the  idea  cf  a language  more  primitive  than  ours 

To  all  this  it  might  be  countered  that  it  has  not  been  shown  that  the  language  facilities  I have 
described  cannot  be  incorporated  in  the  structures  that  SHRDLU  manipulates,  and  that,  even  if 
they  cou’,d  not.  the  work  would  still  be  significant  in  virtue  of  its  original  control  structure  and  its 
dernonstration  that  real  world  knowledge  can  be  merged  with  linguistic  knowledge  in  a working 
whole  Indeed,  although  Winograd  has  not  tried,  m any  straightforward  sense,  to  « ten the 
SHRDLU  system  one  could  say  that  an  extension  of  this  sort  is  being  attempted  by  Brown  (1974) 
with  his  'Believer  System’,  which  is  a hybrid  system  combining  a component  about  beliefs  that  is. 
in  the  sense  of  section  4 below  ’second  generation’,  with  a base  malyser  from  Bruces  Chronos 


system  (1972)  which  is  a rmcro-world-late  first  gener.iticn  -system  in  the  same  sense  as 
Winogiad's.  Others  in  the  last  category  that  should  be  mentioned  are  Davies  and  Isard’s  (1972) 
exploration  of  the  concepts  of  ‘mutt’  and  ‘could’  in  a micro-world  of  tic-tac-toe,  and  Joshi’s 
extension  of  it  (1973),  but  above  all  the  important  and  influential  work  of  Woods  (1972) 

This  work,  most  recently  applied  to  a micro-world  of  lunar  rock  samples,  is  not  discussed  in  the 
detail  It  deserves  in  this  paper  The  system,  based  on  an  augmented  state  transition  network 
grammar,  is  undoubtedly  one  of  the  most  robust  in  actual  use,  in  that  it  is  less  sensitive  to  tlie 
PARTICUF  AR  Input  questions  it  encounters  than  its  rivals.  The  reason  for  not  treating  it  in 
depth  IS  that  both  Woods  and  Winograd  have  argued  in  print  that  their  two  systems  arr 
essentially  equivalent  (Winograd  1971)  (Woods  1973),  and  so,  if  they  are  right,  there  is  no  need  to 
discuss  both,  and  Winograd's  is.  within  the  AI  community  at  least,  the  better  known  of  the  two. 

Their  equivalence  arguments  are  probably  correct:  both  are  grammar-bas?d  deductive  systems, 
operating  within  a question-answering  environment  in  a highly  limited  domain  of  discourse. 
Winogiad’s  system  of  hints  on  how  to  proceed,  within  his  PROGRAMMAR  grammar,  is.  as  he 
himsfV  points  out,  formally  equivalent  to  an  augmented  state  transition  network,  and  in  particular 
to  the  ordering  of  choices  at  nodes  in  Wood’s  system. 

Th'  ' is  a significant  difference  in  their  metaphysical  approaches,  or  presuppositions  about 
meaning  which,  however,  has  no  Influence  on  the  actual  operation  of  their  respective  systems. 
This  ifference  is  disguised  by  the  allegiance  both  give  to  a ’procedural  view  of  meaning’.  The 
difference  is  that  Woods  takes  a much  more  logico-semantic  interpretation  of  the  slogan  than  dor  s 
Winograd.  In  particular,  for  Woods  the  meaning  of  an  input  utterance  to  his  system  is  the 
procedures  within  the  system  that  manipulate  the  truth  conditions  of  the  utterance  and  establisn 
its  tiuth  value 

To  put  the  matter  crudely,  for  iVoods  an  assertion  has  no  meaning  If  his  system  cannot  establish 
Its  truth  or  falsity.  Winograd  lias  certainly  not  committed  himself  to  any  such  extreme  position. 

It  IS  interesting  to  notice  that  Woods’  is,  in  virtue  of  his  s'rong  position  on  truth  conditions, 
probably  the  only  piece  of  work  in  the  field  of  A I and  natura!  language  to  satisfy  Hayes’  (1974) 
recent  demand  that  to  be  "intellectually  respectable"  a knowledge  system  must  have  a natural 
model  theoretic  semantics,  in  Tarski’s  sense.  Since  no  one  has  ever  gi.’en  precise  truth  conditions 
for  any  interesting  piece  of  discoirrse,  such  as,  say.  Woods'  own  papers,  one  might  claim  that  his 
theoretical  presuppositions  necessarily  limit  his  work  to  the  analysis  of  micro-worlds  (as  distinct 
from  everyday  language). 

There  is  a low-level  problem  about  the  equivalence  of  Woods’  and  Winograd’s  systems,  if  we 
consider  what  we  might  call  the  received  common-sense  view  o,‘‘  their  wor’u.  Consider  the 
following  three  assertions: 

(1)  Woods’  system  is  an  implementation  of  a transformational  grammar: 

(2)  Winograd's  work  has  shown  the  irrelevance  of  transformational  grammar  for  language 
analysis-a  view  widely  held  by  the  reviewers  of  his  work; 

(3)  Woods’  and  Winograd’s  systems  are  formally  equivalent-a  view  held  by  both  of  then. 

There  is  clearly  something  of  an  inconsistent  triad  amongst  those  three  widely  held  beliefs.  The 
trouble  probably  centers  on  the  exact  sense  which  Woods’  work  is  formally  equivalent  to  a 
transformational  grammar-not  a question  that  need  detain  us  here,  but  one  worth  pointing  out  in 
passing 
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4.  SOME  MORE  GENERAL  BACKGROUND  ISSUES 


Winograd  J work  is  a central  example  of  the  ‘Artificial  Intelligpn..,  paradigm  of  language’,  using 
■paradigm’  in  Kuhn's  (1970)  sense  of  a large  scale  revision  in  systematic  thinking,  wnere  the 
paradigm  revised  is  the  'generative  paradigm’  of  the  Chomskyan  linguists  (Chomsky  1957).  From 
the  A I point  of  view,  the  generative  linguistic  work  of  the  last  fifteen  years  has  three  principal 
defects.  Firstly,  the  generation  of  sentences,  with  whatever  attached  structures,  is  not  in  any 
interesting  sense  a demonstration  of  human  understanding,  nor  is  the  separation  of  the  well- 
formed,  from  the  ill-formed,  by  such  methods;  for  understanding  requires,  at  the  very  least,  both 
the  generation  of  sentences  as  parts  of  coherent  discourse,  and  some  attempt  to  interpret,  rather 
than  merely  reject,  what  seem  to  be  ill-formed  utterances  Neither  the  transformational 
grammarians  following  Chomsky,  nor  their  successors  the  generative  semanticists  (Lakoff  1071), 
have  ever  explicitly  renounced  the  generative  paradigm. 

Secondly,  Chomsky's  distinction  between  performance  and  competence  models,  and  his  advocacy 
of  the  latter,  have  isolated  modern  generative  linguistics  from  any  effective  test  of  the  systems  of 
rules  It  proposes  Whether  or  not  the  distinction  was  intended  to  :iavt->  his  effect,  it  has  mearit 
that  any  test  situation  necessarily  involves  pe:fo;man.e,  whicii  i'  -onsidered  outside  the  province 
of  serious  linguistic  study.  And  any  embodiment  of  a system  of  roles  in  a computer,  and 
assessment  of  its  output,  would  be  perforriance  A I,  too,  is  much  concerned  with  the  structure  of 
linguistic  processes,  independent  of  any  particular  implementation,  (2)  but  implementation  is  never 
excluded,  as  it  is  from  competence  models,  but  rather  encouraged. 

Thirdly,  as  I mentioned  before,  there  was  until  recently  no  place  in  the  generative  paradigm  for 
interferences  from  facts  and  inductive  generalisations,  even  though  very  simple  examples 
demonstrate  the  need  for  it. 

This  last  point,  about  the  shortcomings  of  conventional  linguistics,  is  not  at  all  new,  and  in  A I is 
at  least  as  old  as  Minsky’s  (1968,  p 22)  observation  that  in  'He  put  the  box  on  the  table.  Because 
It  wasn't  level,  it  slid  off,  the  last  'it'  can  only  be  leferred  correctly  to  the  box,  rather  than  the 
table,  on  the  basis  of  some  knowledge  quite  other  than  that  in  a conventional,  and  implausible, 
linguistic  solution  such  as  the  creation  of  a class  of  'level  nouns’  so  that  a box  would  not  be 
considered  as  being  level. 

These  points  would  be  generally  conceded  hy  those  who  believe  there  is  an  A1  paradigm  of 
language  understanding,  but  'here  would  be  tar  less  agreement  over  the  positive  content  of  the 
paradigm.  The  trouble  begins  with  the  definii'on  of  ’understanding’  as  applied  to  a computer. 
At  one  extreme  are  those  who  say  the  word  can  onlv  refer  to  the  performance  of  a machine;  to  its 
ability,  say,  to  sustain  some  form  of  dialogue  long  enough  and  sensi’oly  enough  for  a human 
interrogator  to  be  unsure  whethv.  what  he  is  conversing  with  is  a machine  or  not.  On  the  other 
hand,  there  are  many,  almost  certainly  a majority  who  argue  that  more  is  required,  in  that  the 
methods  and  representations  of  knowledge  by  which  the  performance  is  achieved  must  be  of  the 
right  formal  sort,  and  that  mere  performance  based  on  ad  hoc  methods  does  not  demonstrate 
understanding. 


(2)  Vide;  "Artificial  Intelligence  is  the  study  of  intellectual  mechanisms  apart  from  applications 
and  apart  from  how  such  mechanisms  are  realised  in  the  human  or  in  animals"  (McCarthy  in 
press) 
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This  issue  IS  closely  related  to  that  ol  the  lole  o(  deduction  in  natural  language  undeistandmg, 
simply  because  deduction  is  often  the  structure  meant  when  'right  methods  are  mentioned.  Tim 
dispute  between  those  who  a'gue  for,  or.  like  Wiiiogiad  use  deductive  methods,  and  those  wha 
advocate  other  inferential  systems  closer  to  common  sense  reasoning,  is  in  man/  ways  a pseud(<- 
issue  because  it  is  so  difficult  to  define  clearly  what  a non-deductive  system  is  (if  by  that  is  meant 
a system  that  cannot  in  principle  be  modelled  by  a deductive  system)  since  almost  any  set  of  formal 
procedures,  including  'invalid  inferences’,  can  be  so  displayed.  The  heart  of  tiie  matter  concerns 
the  most  appropriate  form  of  an  inference  system  rather  than  how  those  inferences  may  be 
a.xiomatised,  and  it  may  well  turn  out  that  the  most  appropriate  form  for  plausible  reasoning  in 
order  to  understand  is  indeed  non -deductive.  This  same  insight  has  largely  defused  another 
heated  issue:  whether  the  r.ppropriate  representations  should  be  procedures  or  declarations. 
Winograd’s  work  was  of  the  former  type,  as  was  shown  by  his  definitions  of  words  like  'pickup'  as 
procedures  for  actually  picking  things  up  in  the  blocks  world.  However  simple,  procedural 
' °sentations  usually  have  the  disadvantage  that,  if  you  are  geing  to  indicate,  for  every  'item’  of 
kt..  'ledge,  hoto  it  is  to  be  used,  then,  if  you  may  use  it  on  a number  of  kinds  of  occi«sions,  you  will 
have  to  store  it  that  nurtiber  of  times.  So.  if  you  want  to  change  it  later,  you  will  also  have  to 
remember  to  ^hange  it  in  ail  the  different  places  you  have  put  it.  There  is  the  additional 
disadvantage  of  lack  of  perspicuity:  anyone  reading  the  prodecural  version  of  the  Winograd 
grammar  rule  I gave  earlier  will  almost  certainly  find  the  conventional,  declarative,  version  easier 
to  understand. 

So  then,  the  fashion  for  all  things  procedural  has  to  some  extent  abated  (see  Winograd  1974). 
There  is  general  agreement  that  any  system  should  show,  as  it  were,  how  it  is  actually  to  be 
applied  to  language,  out  that  is  not  the  same  as  demanding  that  it  should  be  written  in  a 
procedural  language,  like  PLANNER,  i shall  return  to  this  last  point  later. 
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5.  SECOND  GENERATION  SYSTEMS 


To  understand  what  was  n.eant  when  Winograd  contrasted  his  own  with  what  he  called  second 
generation  systems,  ve  have  to  remember,  as  always  in  this  subject,  th.at  the  generations  are  of 
fashion,  not  chronology  or  inheritance  of  ideas  He  described  the  work  of  Simmons,  Schank  and 
myself  among  others  in  his  survey  of  new  approaches,  even  though  the  foundations  and 
terminology  of  those  approaches  were  .set  out  in  print  in  1965,  1967  and  1968  respectively.  What 
those  approaches  and  others  have  in  common  is  the  belief  that  understanding  systems  must  be 
able  to  manipulate  very  complex  linguistic  objects,  or  semantic  structures,  and  that  no  simplistic 
approaches  to  understanding  language  with  computers  will  wo.  k. 

In  an  unpublished,  but  alread)  very  influpptial  recent  paper,  Minsky  (1974)  has  drawn  together 
strands  in  the  work  of  Charm  k (1972)  and  ihe  authors  above  using  a terminology  of  "frames". 

"A  frame  is  a data-structure  for  representing  a stereotype  situation,  like  a certain  kind  of 
living  room,  or  going  to  a children's  birthday  party  .Attached  to  each  frame  are  several  kinds 
of  information.  Some  o'  this  is  information  about  how  to  use  the  frame.  Some  is  about  whar 
to  do  if  those  expectati  rns  are  not  confirmed 

We  can  think  of  a frame  as  a network  of  nodes  and  relations  The  top  levels  of  a frame  ai\- 
fixed  and  represent  things  that  ate  always  true  about  the  supposed  situation.  The  lower  I'^vels 
have  many  terminals  --  "slots"  that  must  be  filled  by  specific  instances  or  data.  Each  terminal 
can  specify  ccaditions  its  assignments  must  meet.  ..  Simple  conditions  are  specified  by  markers 
that  might  require  a terminal  assignment  to  be  a person,  an  object  of  sufficient  value,  etc..." 

The  key  point  about  such  structures  is  that  they  attempt  to  specify  in  advance  what  is  going  to  b - 
said  and  how  the  world  encountered  is  going  to  be  .structured.  The  structures,  and  the  inference 
rules  that  apply  to  them,  are  also  expressions  of  'partial  information’  (in  McCarthy’s  phrase)  that 
are  not  present  in  first  generation  systems.  As  I showed  earlier  with  thi  ‘women  and  soldier.s 
example',  such  loose  inductive  information,  seeking  confirmation  from  the  surrounding  context,  is 
required  for  very  simnie  sentences.  In  psychological  and  visual  terms,  frame  approaches  envisage 
an  understander  as  at  least  as  much  a looker  as  a seer. 

I shall  now  de.cribe  briefly  five  approaches  that  might  be  called  second  generation. 

Chariiiak 

The  new  work  which  owes  most  to  Minsky's  advocacy  is  Charniak's  He  studied  what  sorts  of 
inferential  information  (Charniak  197'’,  1973.  I93J)  would  be  needed  to  resolve  pronoun 
ambiguities  in  children's  stories,  and  that  sense  to  understand  them.  One  of  his  example 
'stones'  IS:  'Jack  was  invited  to  jane’s  urthday  party  She  wondered  if  he  would  like  a kite.  A 
fiiend  told  Jane  that  jack  already  had  '.ite,  and  that  he  wc  make  her  take  it  back.’ 

The  problem  concerns  the  pen  .inmate  word  "it",  and  deciding  hf  <=  it  refers  to  the  first  kite 
mentioned  or  the  second. 

Charniak's  analysis  begins  by  pointing  out  that  a great  deal  of  what  is  required  to  understand 
that  story  is  implicit:  know!'*dge  about  the  giving  of  presents,  knowledge  that  if  one  possesses  one 
of  a certain  sort  of  thing  then  one  may  well  not  want  another  and  so  on 
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Charniak’s  system  does  not  actually  run  as  a program,  but  is  a theoretical  structure  of  rules  called 
'demons’  that  correspond  roughly  to  what  Minsky  later  called  frames  A demon  for  this  example 
would  be,  ‘If  we  see  that  a person  might  not  like  a present  X,  then  look  for  X being  returned  to 
the  store  where  it  was  bought'. 

If  we  see  that  happening,  or  even  being  suggested,  assert  that  the  reason  why  is  * ..  ? does  not 
li»  X. 

The  important  words  there  are  “look  for”,  which  suggest  that  there  may  well  be  confirming  hint 
found  in  the  story  and.  if  there  are,  then  this  tentative,  partial  inference  is  correct,  and  we  have  a 
definite  and  correct  answer.  This  approach  of  using  partial  (not  necessarily  true)  inferences,  in 
order  to  assert  a definite  answer,  is  highly  characteristic  of  "second  generation"  systems. 

The  demons  are,  as  with  Winograd’s  work,  expressed  in  a procedural  language  which,  on 
running,  will  seek  for  a s 'ccession  of  interrelated  "goals". 

Here  for  example,  is  a demon  concerned  with  another  story,  about  a child’s  piggy  bank  (PB)  and 
a child  shaking  it  looking  for  money  ar  ' ing  no  sound.  The  demo,.  7B-CUT-OF,  is 
formalised  as: 


(DEMON  PB-OJT-OF 

(NOLD  PB  PERSON  M N) 

(?N  OUT-OF  ’N  ?PB) 

(GOAL  (?  IS  ?PB  PIGGY-BANK)) 

(GOAL  C IS  ?M  MONEY)  8DEDUCE)) 

(GOAL  (?NOLD  SHAKE  ?PERSON  ?PB)8TRUE) 
(ASSERT  (?  HAVE  ? PERSON  ?M)) 

(ASSERT  (?  RESULT  ?N  ?NOLD))) 


.Again,  it  is  not  necessary  to  explain  the  notation  in  detail  to  see  that  conditions  are  being  stated 
for  the  contents  of  a piggy  bank  having  been  emptied.  The  pattern  being  sought  by  the  demon 
in  operation  is  the  third  line.  If  a chain  of  demons  can  "reach"  one  of  the  possible  referents  in  a 
story  then  there  is  a success  registered  and  the  ambiguity  of  the  corresponding  pronoun  is 
resolved. 

It  ca.T  be  seen  that  the  information  encoded  m the  system  is  of  a highly  specific  sort in  the 

present  case  it  is  not  about  containers  as  such,  and  how  to  get  the^  contents  out,  but  about  Pi^p;y 
Banks  in  particular,  and  relies  on  that  particular  knowledge  having  been  put  in.  Not  all  the 
knowledge  is  of  this  general  sort:  in  a recent  paper  (Charniak  1974)  where  the  "kite"  story  is 
reconsidered  there  are  rules  of  considerable  generality  and  interest.  One  such  is  what  Charniak 
calls  a R*SSA  rule: 

"If  the  story  gives  information  which  would  make  it  plausible  to  infer  that  PERSON  is 

favorably  inclined  towards  action  A.  and  PERSON  does  S,  a significant  subaction  (SSA)  of 

A.  then  infer  that  PERSON  is  doing  A." 

An  important  assumption  of  Charniak’s  is  what  I shall  call  his  "decoupling  assumption”,  name'v 
that  this  work  on  the  knowledge  based  determination  of  reference  can  be  effectively  decorif  eo 
from  any  particular  syntactic  or  semantic  representation,  or  of  its  application,  and  can  be  explored 
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in  isolation.  Thus,  the  apparently  English  v;ords  in  the  PB-OUT-CF  demon  ~bove  do  not  have 
any  particular  status  other  than  that  of  English  words  that  happen  to  occur  in  the  stories  being 
examined.  1 shall  retui.T  to  this  assumption  later. 


Colby 

Colby  (Colby  et  al  l97lXEnea  and  Colby  l973)(Colby  and  Parkison,  in  press)  has  developed  over 
a period  of  several  years  an  on-line  dialogue  system  that  simulates  a paranoid  patient  called 
PARRY.  A measure  of  its  success  is  that  it  is  perhaps  the  most  used  A.I.  program  ii  existence; 
used,  that  is  . by  those  outside  the  research  project  that  developed  it.  Here  is  a discontinous 
sample  of  its  output,  again  with  the  systems  responses  in  upper  case: 


Have  you  been  hospitalized  before? 

THIS  IS  THE  FIRST  TIME 
How  long  have  you  been  there? 

ABOUT  TWO  WEEKS. 

Any  headaches? 

MY  HEALTH  IS  FINE. 

Are  you  having  any  difficulty  w th  your  memory? 

NO 

Then  perhaps  you  can  tell  me  who  is  the  President  of  the  United  States? 

NIXON  IS  PRESIDENT. 

What  day  is  today? 

TODAY  IS  THURSDAY 
What  season  is  it? 

LETS  GET  ON  WITH  THE  INTERVIEW. 

Colby’s  aim  was  to  produce  a system  that  was  not.  in  his  word,  "fragile"  in  the  way  that  most  A.I. 
systems  are:  oi  that  did  not  break  down,  or  give  up,  if  the  input  was  not  exactly  what  it 
expected;  but  one  that  carried  on  somehow  in  all  circumstances,  in  the  way  that  conversing 
humans  do. 

Fragility  is  avoided  by  havin^  no  conventional  syntax  analyser,  and  by  not  even  attempting  to 
take  account  of  all  the  words  i i the  input.  Tins  is  a considerable  aid,  since  any  parser  that  begins 

to  parse  a more  than  usually  polite  request  such  as  "Would  you  be  so  kind  as  to ” is  going  to  be 

in  trouble.  British  English  speakers  arriving  in  the  U.S.  quickly  learn  to  delete  such  phrases  since 
they  cause  great  confusion  to  human  listeners  in  store' 

The  input  text  is  segmented  by  a heuristic  that  breaks  It  at  any  of  a range  of  key  words.  Patterns 
are  then  matched  with  each  segment.  There  are  at  present  about  1700  patterns  on  the  list 
(Colby  and  Parkison.  in  press)  that  are  stored  and  matched  not  against  any  syntactic  or  semantic 
representations  of  words  (ex^'ept  to  deal  with  contractions  and  misspellings)  tut  against  the  input 
word  string  directly  and  by  a process  of  sequential  deletion.  So,  for  example,  "What  is  your  main 
problem"  has  a root  verb  "Be"  substituted  to  become 
WHAT  BE  YOU  MAIN  PROBLEM. 

It  is  then  matched  successively  in  the  following  forms  after  deletion: 

BE  YOU  MAIN  PROBLEM 
WHAT  YOU  MAIN  PROBLEM 


WHAT  BE  MAIN  PROBLEM 
WHAT  BE  YOU  PROBLEM 
WHAT  BE  YOU  MAIN 
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and  only  the  penuhimate  line  exists  as  one  of  the  stored  patterns  and  so  is  matched.  Stored  in  the 
same  format  as  the  patterns  are  rules  expressing  the  consequences  for  the  "patient"  of  detecting 
aggression  and  over-friendliness  in  the  interviewer's  questions  and  remarks.  The  matched 
patterns  found  are  then  tied  directly,  or  via  these  inference  rules,  (o  response  patterns  which  are 
generated. 

Enoimoiis  ingenuity  has  gone  into  the  heuristics  of  this  system,  as  its  popularity  testifies.  The 
system  has  also  changed  considerably:  it  is  now  called  PARRY2  and  contains  the  above  pattern 
matching,  rather  than  earlier  key  word,  heuristics.  It  has  the  partial,  or  what  some  would  call 
"pragmatic",  rules  about  expectation  and  intention,  and  these  alone  might  qualify  it  as  "second 
generation"  on  some  interpretations  of  the  phrase.  A generator  is  also  being  installed  to  avoid  the 
production  of  only  "canned"  responses 

Colby  and  his  asscKiates  have  put  considerable  energy  into  actually  trying  to  find  out  whether  or 
not  psychiatrists  can  distinguish  PARRY’s  responses  from  those  of  a patient  (Colby  and  Hilf 
1973).  This  is  probably  the  first  attempt  actually  to  apply  Turing’s  test  of  machine-person 
distiiigiiishability  There  are  statistical  difficulties  about  interpreting  the  results  but,  by  and  large, 
the  result  is  that  the  sample  questioned  cannot  distinguish  the  two  Whether  or  not  this  will 
influence  those  who  still,  on  principle,  believe  that  PARRY  is  not  a simulation  because  it  "does 
not  understand",  remains  to  be  seen.  It  might  be  argued  that  they  are  in  danger  of  falling  into  a 
form  of  Papert's  "human-superhuman  fallacy’  of  attacking  machine  simulations  because  they  do 
not  peform  superhuman  tasks,  like  translating  poetry,  tasks  that  some  people  certainly  can  do. 
When  such  sceptics  say  that  PARRY  does  not  understand  they  have  in  their  minds  a level  of 

understanding  that  is  certainly  high one  could  extend  their  case  ironically  by  pointing  out  that 

very  few  people  understand  the  content  of  sentences  in  the  depth  and  detail  that  an  analytic 
philosopher  does,  and  a very  good  thing  too.  There  can  be  no  doubt  that  many  people  on  many 
occasions  DO  seem  to  understand  in  the  way  that  PARRY  does. 


The  remaining  three  systems  differ  from  the  two  above  in  their  attempt  to  provide  some 
representational  structure  quite  different  from  that  of  the  English  input.  This  means  th.  use  of 
cases,  and  of  complex  structures  that  allow  inferences  to  be  drawn  from  the  attribution  of  case  in 
wajS  I shall  explain.  There  is  also,  in  the  remaining  systems,  some  attempt  to  construct  a 
primitive  or  reduced  vocabulary  into  which  the  language  represented  is  squeezed. 


Siininoiis 

Simmons’  work  is  often  thought  of  as  a "memory  model"  though  he  does  in  fact  pay  more 
attention  to  word  sense  ambiguity,  and  to  the  leactua!  recognition  in  text  than  do  many  other 
authors  For  him  the  fundamental  notion  is  that  of  a "semantic  network",  defined  essentially  by 
the  statement  of  relational  triples  of  form  aRb,  where  R is  the  name  of  a relation  and  a and  b are 
the  names  of  nodes  in  the  network.  SimoKins  work  with  this  general  formalism  goes  bark  to  at 
least  1966  (Simmons  et  al,  1966)  but,  in  its  newer  form  with  case  formalism,  it  has  been  reported 
since  1970  (Simmons  1970b)  (Simmons  and  Bruce  1971)  (Simmons  and  Slocum  1972).  (Simmons 
1973),  and  (Hendrix  et  al  1973)  may  reasonably  be  considered  a further  implementation  of 
Simmons’  methods. 


Simmons  considers  the  example  sentence  "John  broke  the  window  with  a hammer"  This  is 
analysed  into  a network  of  nodes  C1.C2,C3,C'4  corresponding  to  the  appropriate  senses  of  "John", 
"break",  "Window"  and  "Hammer"  respectively.  The  linkages  between  the  nodes  are  labelled  by 
one  of  the  following  "deep  case  relations":  CAUSAL-ACTANT  (CAl,  CA2),  THEME,  LOCUS, 
SOURCE,  and  GOAL.  Case  relations  are  specifications  of  the  way  dependent  parts  of  a 
sentence,  or  concepts  corresponding  to  parts  of  a sentence,  depend  on  the  main  action.  So,  in  this 
case.  John  is  the  first  causal  actant  (CAl)  of  the  breaking,  the  hammer  is  considered  the  second 
causal  actant  (CA2)  of  that  breaking,  and  the  window  is  the  theme  of  the  breaking.  Thus,  the 
heart  of  the  analysis  could  be  represented  b«  a diagram  as  follows: 


"John"  C2 


or  by  a set  of  relational  triples; 

(Cl  CAl  C2XC1  CA2C4XCI  THEME  C3) 

r(owever,  this  is  not  the  full  representation,  because  my  addition  of  the  word  labels  to  the  diagram 
is  misleading,  since  the  nodes  are  intended  to  be  names  of  senses  of  words,  related  to  the  actual 
occurrence  of  the  corresponding  word  in  a text  by  the  relation  TOK  (for  token)  In  an 
implementation,  a node  would  have  an  arbitrary  name,  such  as  L97,  which  would  name  a stored 
sense  definition.  So,  for  a sense  of  "apple"  Simmons  suggests  an  associated  set  of  features;  NBR- 
singulars  SHA PE-spherical,  COLOR-red,  PRINTlMACE-apple,  THEME>:'-eat,  etc.  If  the  name 
of  the  node  tied  to  this  set  of  features  was  indeed  L97,  then  that  node  might  become,  say,  C5  on 
being  brought  into  some  sentence  representation  during  a parsing  Thus,  the  diagram  I gave 
must  be  thought  to  be  supplemented  by  other  relational  ties  from  the  nodes;  so  that  the  full 
sentence  about  John  would  be  represented  by  the  larger  set  of  triples; 


(Cl  TOK  break)(Cl  CAl  C2)(CI  THEME  C3)(C1  CA2C4) 

(C2  TOK  John)(C2  DET  DefXC2  NBR  S) 

C3  TOK  WindowXC3  OET  DefXC3  NBR  S) 

(C4  TOK  Hammer)(C4  DET  lndefXC4  NBR  SXC4  PREP  With) 


Word  sense  ambiguity  :s  taken  account  of  in  that  the  node  for  one  sense  of  "hammer"  would  be 
different  from  that  corresponding  to  some  other  sense  of  the  same  word.,  such  as  that  meaning 
Edward.  Hammer  of  the  Scots,  to  take  a slightly  strained  (3)  alternative  for  this  sentence. 


(3)  Simmons  normal  example  '•f  word  sense  ambiguity  does  not  apply  to  the  sentence  above;  he 
distinguishes  "pitcher I*.  a pouring  container,  from  "pitcher2",  in  the  US.  sense  of  one  who  bowls 
a ball. 


H 


The  network  above  is  also  a representajion  nf  the  following  sentence,  which  can  be  thought  of  as 
surface  variants  of  a single  ”ur  dei  lying"  structure: 

John  broke  the  window  with  a hammer 
John  broke  the  hammer 
The  hammer  broke  the  window 
The  window  broke 

Not  all  parts  of  that  network  will  be  set  up  by  each  of  these  sentences,  of  course,  but  the  need  for 
some  Item  to  fill  an  appropriate  slot  can  be  inferred;  i.e.  of  the  first  causal  actant  in  the  last  two 
sentences  The  sentences  above  are  recognized  by  neans  of  the  "ergative  paradigm"  of  ordered 
matching  patterns,  of  which  the  following  list  is  a pait- 

(CAI  THEME  CA2) 

(CAI  THEME) 

(CA2  THEME) 

(THEME) 

These  sequences  will  each  match,  as  left-right  ordered  items,  one  of  the  above  sentences.  It  will  be 
clear  that  Simmons’  method  of  ascribing  a node  to  each  word-sense  is  not  in  any  way  a primitive 
system,  by  which  I mean  a system  of  classifiers  into  which  all  word  senses  are  mapped. 

Simmons  is  however,  considering  a system  of  paraphrase  rules  that  would  map  from  one  network 
to  another  in  a way  that  he  claims  is  equivalent  to  a system  of  primitives.  Thus,  in  (Simmons 
1973)  he  considers  the  sentences. 

John  bought  the  boat  from  Mary 
Mary  sold  the  boat  to  John 

which  would  normally  be  considered  approximate  paraphrases  of  each  other.  He  then  gives 
"natural"  representations,  in  his  system,  as  follows  in  the  same  order  as  the  sentences: 

Cl  TOK  buy,  SOURCE  (Mary).  GOAL  (John),  THEME  (boat) 

Cl  TOK  sell,  SOURCE  (Mary),  GOAL  (John),  THEME  (boat) 

and  also  the  single  representation  for  both  sentences,  as  below,  using  a primitive  action  "transfer" 
(see  description  of  Schank’s  work  in  next  section)  as  follows. 

Cl  TOK  and.  ARGS  C2. 

C2  TOK  transfer.  SOURCE  (John).  GOAL  (Mary).  THEME  (money) 

C3  TOK  transfer.  SOURCE  (Mary),  GOAL  (John),  THEME  (boat) 

Simmons  opts  for  the  first  form  of  representation,  given  the  possibility  of  a transfer  rule  going 
from  either  of  the  shallower  representations  to  the  other,  while  in  (Hendrix  et  al  1973),  the  other 
approach  is  adopted,  using  a primitive  action  EXCHANGE  instead^  of  'transfer  . The 
implementation  under  construction  .s  a front-end  parser  of  the  Woods  augmented  transition 
network  type  (see  Woods  1970),  and  a generation  system  going  from  the  semantic  networks  to 
surface  strings  described  in  detail  in  (Simmons  and  Slocum  1972).  Simmons  has  also  given 
considerable  time  (Simmons  and  Bruce  l97i)to  the  automatic  translation  of  the  networks  into  a 
corresponding  first-order  predicate  calculus  fo-mat  of  the  sort  developed  by  Sandewall  (I97U. 
This  last  IS  particulaiy  valuable  because,  if  generalisable,  it  shows  that  any  linguistic  codings  in 
network  form  ran  be  translated  into  some  form  of  the  predicate  calculus,  if  that  formalism  and  its 
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asso<’iated  proof  techniques  can  be  shown  to  be  appro-priate  for  any  problem^  in  the  area  of 
natural  language  analysis. 


Scliaitk 

Schank’s  is  a rich  system  of  semantic  representation,  developed  over  a period  of  six  years,  with  the 
collaboration  of  a number  of  talented  student.^.  Its  graph  system  of  notation  has  influenced 
psychlogists  like  Abelson  (197.'’.).  Schank’s  contribution  has  been  the  notational  system 
representing  the  structure  cf  natural  language  sentences,  and  this  has  been  programmed  by 
various  collaborators  over  th  ' /ears  In  its  present  version,  called  MARGIE  (Schank  et  al  1973)  it 
has  an  analyser  of  English  due  to  Riesbeck  (197‘1).  a semantic  memory  component  due  to  Rieger 
(1974),  and  a generator  of  English  due  to  Goldman  (1974).  MARGIE  produrps  output  in  two 
modes,  demonstrating  the  sort  of  conceptual  infeiencing  that  goes  on  at  the  level  of  the  semantic 
representations:  (4)  the  PARAPHRASE  mode  and  the  INFERENCE  mode.  Samples  of  input 
and  output  to  and  from  the  two  modes  can  be  shown  thus: 

(INFERENCE  MODE) 

INPUT:  John  told  Mary  that  Bill  wants  a book. 

OUTPUT:  A BOOK  ABOUT  WHAT? 

Mary  knows  that  Bill  wants  a book. 

Pill  wants  to  come  to  have  a book. 

Bill  wants  someone  to  cease  to  have  a book. 

Bill  wants  to  read  a book. 

(PARAPHRASE  MODE) 

INPUT:  John  killed  Mary  by  choking  Mary. 

OUTPUT:john  stangled  Mary. 

John  choked  Mary  and  she  died  because  she  could  not  breathe. 

Mary  died  because  she  was  unable  to  inhale  some  ai  ,d  she  was  unable 
to  inhale  some  air  because  John  grabbed  her  neck 

The  aim  of  Schank’s  system  has  always  been  to  provide  a representation  of  meaning  in  terms  of 
which  these  and  other  tasks,  such  as  machine  translation,  could  be  carried  out;  a representation, 
moieover,  that  was  to  be  independent  of  any  particular  languages,  and  of  syntax  ard,  indeed  of 
all  surface  structure  whate/er. 

The  structure  of  Schank’.s  graphs  is  that  of  dependency  grammei  (Hays  1964)  and  the  items  in  the 
graph  are  of  four  types,  or  conceptual  categories.  They  are  symbolized  as  PP,ACT,PA  and  AA, 
which  are  acronyms,  but  which  correspond  closely  (for  the  purpose  of  understanding  then 
fjn'tion)  to  those  of  noun,  verb,  adjective  and  adverb,  respectively.  The  basic  structure  is  called 
a conctptualisation,  and  is  normally  introduced  with  a straightforward  dependency  structure  such 
as.  for  the  sentence  "The  man  took  a book": 

P 0 


(4)  .Schank  distinguishes  "conceptual"  and  "semantic"  representations  in  a way  that  is  important 
for  him  within  his  own  system.  However,  I shall  use  the  terms  indifferently  since,  in  this  brief 
and  superficial  description,  nothing  hangs  upon  the  distinction. 
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man  ^ takt  ♦-  book 

Here  "p"  indicates  past,  and  ft  is  the  dependency  symbol  linking  a PP  to  the  ACT  ("take”)  which 
is  the  hub  of  the  conceptualisation,  as  with  Simmons.  The  "o"  indicates  the  objective  case, 
marking  the  dependence  of  ihe  object  PP  on  the  central  ACT.  There  is  a carefully  constructed 
syntnx  of  linkages  between  the  conceptual  categories,  that  will  be  described  only  in  part  in  what 
follows. 

The  next  stage  of  the  nofation  involves  an  extended  case  notation  and  a set  of  primitive  ACT’s 
as  well  as  a number  of  items  such  as  PHYSCONT  which  indicate  other  states,  and  items  of  a 
fairly  simplified  psychological  theory  (the  dictionary  entry  for  "advise"  for  example,  contains  a 
subgraph  telling  us  that  Y is  "pleased"  as  part  of  the  meaning  of  "X  advises  Y"  (Schank  1973)). 
There  are  four  cases  in  the  system,  and  their  subgraphs  are  as  follows: 


0 

Objective  case:  ACT  <-  PP 


R —PP 

Recipient  case:  ACT  *-1 

-PP 


Instrumental  case: 


0 -PP 

Directive  case:  ACT  •-) 

— PP 

There  are  at  present  fourteen  basic  actions  forming  the  nubs  of  graphs,  as  well  as  a det  ult  action 
DO.  They  are:  PROPEL.  MOVE.  INGEST.  EXPEL.  GRASP,  PTRANS,  Iv.TRANS, 

ATRANS,  SMELL,  LOOK-AT,  LiSTEN-TO,  CONC  and  MBUILD.  The  notions  of  case  and 
primitive  act  are  related  by  rules  in  the  development  of  conceptualisations.  So,  for  example,  the 
primitive  act  INGEST  has  as  its  instrument  the  act  PTRANS.  There  are  also  other  inferences 
from  any  ACT  classified  as  its  INGEST  action,  such  as  that  the  thing  ingested  changes  its  form; 
that  if  the  thing  ingested  is  edible  the  ingestcr  becomes  "more  nourished"  etc.  (see  Schank  1973  pp. 
38ff.).  This  will  all  become  clt?rer  if  we  consider  the  transition  trom  a dictionary  entry  for  an 
action  to  a filled  in  conceptualisation.  Here  is  the  dictionary  entry  for  the  action  "shoot": 


0 D — Y 

PROPEL  - bullet 
•<ih  ♦--gun 

Y^  hurt 


We  can  consider  this  entry  as  an  active  "frame-like"  object  seeking  filler  items  in  any  context  in 
which  It  is  activated.  Thus,  in  the  sentence  "John  shot  the  gir!  with  a rifle",  the  variables  will  be 
filled  in  from  context  and  the  case  inference  will  be  made  from  the  main  act  PROPEL,  which  is 
that  its  instrument  is  MOVE,  GRASP  or  PROPEL,  and  so  vie  will  arrive  at  the  whole 
conceptualisation: 


iK-r 
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o ■•girl  1 rifle 

John  4=^  PROPEL  I let  <-1  •- 

bullet'flT’  - rifle  PROPEL 

t <|$>PHySC0NT  to 

girl  ou 1 1 e t 

to 

t i 

rifle  girl 


This  case  reference  must  be  made,  according  to  Schank,  in  order  to  achieve  an  adequate 
representation.  There  is,  in  the  last  diagram,  a certain  redundancy  of  expression,  but  as  v»e  shall 
see  in  the  next  section  this  often  happens  with  deeper  semantic  notations. 

More  recently,  Schank,  together  with  Rieger,  has  developed  a ncv<  class  of  causal  inferences  which 
deepen  the  diagrams  still  further.  So,  in  the  analysis  of  "John’s  cold  improved  because  I gave  him 
an  apple"  in  (Schank  1974a)  the  extended  diagram  contains  at  least  four  yet  lower  levels  of  causal 
arrowing,  including  one  corresponding  to  the  notion  of  John  constructing  the  idea  (MBUILD) 
that  he  wants  to  eat  an  apple  So  we  can  see  that  the  underlying  explication  of  meaning  here  is 
not  only  in  the  sense  of  linguistic  primitives,  but  in  terms  of  a theory  of  mental  acts  as  well. 

Now  there  are  a number  of  genuine  expositional  difficulties  here  for  the  commentator  faced  with 
a system  of  this  complexity.  One  aspect  of  this  is  the  stages  of  development  of  the  system  itself, 
which  can  be  seen  as  a consistent  process  of  producing  what  was  argued  for  in  advance.  For 
example,  it  was  acclaimed  early  on  to  be  a system  of  semantic  structure  underlying  the  "surface  of 
natural  language",  although  initially  there  were  no  primitives  at  all,  and  as  late  as  (Schank  et  al 
1970)  there  was  only  a single  primitive  TRANS,  and  most  of  the  entries  in  the  dictionary 
consisted  of  the  English  words  coded  together  with  subscripts.  Since  then  the  primitive  system 
has  blossomed  and  there  are  now  fifteen  primitives  for  ACTS,  including  three  for  the  original 
TRANS  Itself 

Each  exposition  of  the  system  recounts  its  preceding  phases,  from  the  original  primitive-free  one, 
through  to  the  present  causal  inference,  rather  as  each  human  foetus  is  said  to  relive  in  the  womb 
all  the  evolutionary  stages  of  the  human  race  The  orly  trouble  with  this,  from  an  outsider’s 
point  of  view,  is  that  at  each  stage  the  representation  has  been  claimed,  in  firm  tones,  to  be  the 
correct  one.  while  at  the  same  time  Schank  admits,  in  moments  of  candour  (Schank  1973)  that 
there  is  no  end  to  the  conceptual  diagramm  r.g  of  a sentence.  This  difficulty  may  well  reflect 
genuine  problems  in  language  itself,  and  in  its  acutest  form  concerns  the  three  way  distinction 
between  an  attractive  notation  for  displaying  the  "meanings  of  words",  the  course  of  events  in  the 
real  world,  and,  finally,  analysis.  It  is  noi  always  clear  whether  or  not  procedures  implementing 
conceptual  dependency  are  intended  to  recapture  all  the  many  phases  of  expansion  of  the 
diagrani,. 

This  raises  the,  to  me,  important  question  of  the  application  of  a semantic  system,  that  1 shall 
touch  on  again  later.  Schank,  for  example,  does  mention  in  passing  the  questions  of  word-sense 
ambiguity,  and  the  awful  ambiguity  of  English  prepositions,  but  they  are  in  no  wa^  central  for 
him,  and  he  assumes  that  with  the  availability  of  "the  correct  representation",  his  system,  when 
implemei,  d,  must  inevitably  solve  these  traditional  :;r.J  vexing  questions.  No  procedures  are 
hinted  at  along  with  the  graphs  as  to  how  this  is  to  be  done.  A distinction  of  importance  may  be 
coming  apparent  here  between  Schank’s  work  and  Rieger's:  in  Rieger’s  thesrs  (Rieger  1374)  the 
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rules  of  inference  appear  to  create  separate  and  new  subgraphs  which  may  stancJ  in  an  inferential 
relation  to,  say,  pronoun  reference,  etc.  But  in  Schank’s  corresponding  papers  the  same 
inferences  are  not  applied  to  actual  pioblems  (Schank  1974a)  but  simply  complicate  the  conceptii.a! 
graphs  further.  Closely  connected  with  this  is  the  question  of  the  survival  of  ;he  surface  structure 
III  the  diagrams  Until  very  recently  primitivisation  applied  only  to  verbs,  that  of  nouns  being  left 
to  Weber  1972).  Most  recently,  though,  noun  words  have  been  disappearing  from  diagrams  and 
have  been  replaced  by  categories  such  as  :PHYSOBS  But  it  is  clear  that  the  surface  is  only 
slowly  disappearing,  rather  than  having  been  abhorred  all  along. 

In  his  most  recent  publication  (Schank  1974b)  there  are  signs  that  this  trend  of  infinitely 
proliferating  diagrams  is  reversing  In  it  Schank  cjnsiders  the  application  of  his  approach  to  the 
representation  of  text,  and  concludes,  correctly  in  my  view,  that  the  representations  of  parts  ot  *.he 
text  must  be  interconnected  by  causal  arrows,  and  that,  in  order  to  preserve  lucidity,  the 
conceptual  diagrams  for  individual  sentences  and  their  parts  must  be  abbreviated,  as  by  triples 
such  as  PEOPLE  PTRANS  PEOPLE.  Here,  indeed,  the  surface  simply  has  to  survive  in  the 
representation  unless  one  is  prepared  to  commit  oneself  to  the  extreme  view  that  the  ordering  of 
sentences  in  a text  is  a purely  superficial  and  arbitrary  matter.  The  sense  in  which  this  is  a 
welcome  reversal  of  a trend  should  be  clear,  because  in  the  "causation"  infeience  development 
mentioned  earlier,  all  the  consequences  and  effects  of  a conceptualisation  had  to  be  drawn  within 
Itself.  Thus,  in  the  extreme  case,  each  sentence  of  a text  should  have  been  represented  by  a 
diagram  containing  most  or  all  of  the  text  of  which  it  was  a part.  Thus  the  representation  of  a 
text  would  have  been  impossible  on  such  principles. 


Wilks 

My  own  system  also  has  a uniform  representation,  in  terms  of  structures  of  primitives,  for  the 
content  of  natural  language  It  is  uniform  in  that  information  that  might  conventionally  be 
considered  syntactic,  semantic  or  factual  is  all  represented  within  a single  structure  of  complex 
entities  call'd  templates,  while  these  are  in  turn  constructed  from  a budget  of  8o  primitive 
semantic  entities. 

The  system  runs  on-line  as  a package  of  LISP  and  MLISP  programs,  taking  as  input  small 
paragraphs  of  English  that  can  be  made  up  by  the  user  from  a vocabulary  of  about  600  words, 
and  producing  a good  French  translation  as  output.  This  environment  provides  a pretty  clear  test 
of  language  and  understanding,  because  French  translations  for  everyday  prose  are  either  right  or 
wrong,  and  can  be  seen  to  be  so,  while  at  the  same  time,  the  major  difficulties  of  understanding 
programs-  word  sense  ambiguity,  case  ambiguity,  di.*^icult  pronoun  references,  etc.-ran  all  be 
represented  within  a machine  translation  environment  b;  for  example,  choosing  the  words  of  the 
input  sentence  containing  a pronoun  reference  difficulty  so  that  the  possible  alternative  references 
have  different  genders  in  French  In  that  way  the  French  output  makes  quite  clear  whether  or 
not  the  program  has  made  the  correct  inferences  in  order  to  understand  what  it  is  translating. 
The  program  is  reasonably  robust  in  actual  performance  and  will  even  tolerate  a certain  amount 
of  bad  grammar  in  the  input,  since  it  is  not  performing  a syntax  analysis  in  the  conventional 
sense,  but  seeking  messages  representable  in  the  semantic  structures  employed. 

Typical  input  would  be  a sentence  such  as  "John  lives  out  of  town  and  drinks  his  wine  out  of  a 
bottle.  He  then  throws  the  bottles  out  of  the  window."  The  program  will  produce  French 
sentences  with  different  output  for  each  of  the  three  occurrences  of  "out  of",  since  it  realises  that 
they  function  quite  differently  on  the  three  occasions  of  use,  and  that  the  difference  must  be 
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ri.’flected  in  the  French.  A sentence  such  as  "Give  the  monkeys  bananas  although  they  are  not 
ripe  because  they  are  very  hungry"  produces  a translation  with  different  equivalents  for  the  two 
occurences  of  "they",  because  the  system  correctly  realises,  from  what  I shall  describe  below  as 
preferencf  considerations,  that  the  most  sensible  interpretation  is  one  in  which  the  Hrst  "they" 
refers  to  the  bananas  and  the  second  to  the  monkeys,  and  bananas  and  monkeys  have  different 
genders  in  French.  These  two  examples  are  dealt  with  in  the  "basic  mode"  of  the  system  (Wilks 
|97?a).  In  many  cases  it  cannot  resolve  pronoun  ambiguities  by  the  sort  of  straightforward 
"preference  considerations"  used  in  the  last  example,  where,  roughly  speaking,  "ripeness"  prefers  to 
be  predicated  of  plant-like  things  and  "hunger"  of  animate  things.  Even  in  a sentence  as  simple 
as  "John  drank  the  wine  on  the  table  and  it  was  good  ",  such  considerations  are  inadequate  to 
resolve  the  ambiguity  of  "it"  between  wine  and  table,  since  both  may  be  good  things.  In  such 
cases  of  inability  to  resolve  within  its  basic  mode,  the  program  prints  COMMON  SENSE 
INFERENCES  CALLED  on  the  screen  and  deepens  the  representation  of  the  text  so  as  to  try 
and  set  up  chains  of  inference  that  will  reach,  and  so  prefer,  only  one  of  t''e  possible  referents.  I 
will  return  to  these  processes  in  a moment,  but  first  I shall  give  some  brief  description  of  the  basic 
representation  set  up  for  English. 

For  each  sense  of  a word  in  its  dictionary  the  program  sees  a formula.  This  is  a tree  structure  of 
semantic  primitives,  and  is  to  be  interpreted  formally  using  dependency  relations.  The  main 
element  in  any  formula  is  the  rightmost,  called  its  head,  and  that  is  the  fundamental  category  to 
which  the  formula  belongs.  In  the  formulas  for  actions,  for  example,  the  head  will  always  be  one 
of  the  primitives,  PICK.  CAUSE.  CHANCE.  FEEL.  HAVE.  PLEASE.  PAIR,  SENSE.  USE. 
WANT,  TELL.  BE.  DO.  FORCE.  MOVE.  WRAP.  THiNK,  FLOW,  MAKE.  DROP.  STRIK, 
FUNC.  or  HAPN. 

Here  is  the  tree  structure  for  the  action  of  drinking: 


Once  again,  it  is  not  necessary  to  explain  the  formalism  in  any  detail,  to  see  that  this  sense  of 
"drink"  is  being  expressed  as  a causing  to  move  a liquid  object  (FLOW  STUFF)  by  an  animate 
agent,  into  that  same  agent  (containment  case  indicated  by  IN.  and  formula  syntax  identifies 
SELF  with  the  agmt)  and  via  (direction  case)  an  aperture  (THRU  PART)  of  the  agent. 

Template  structures,  which  actually  represent  sentences  and  their  parts  are  built  up  as  netwoiks  of 
formulas  like  the  one  above.  Templates  always  consist  of  an  agent  node,  an  action  node  and  an 
object  node,  and  other  nodes  that  may  depend  on  these  So,  in  building  a template  for  "John 
drinks  wine",  the  whole  of  the  above  tree-formula  for  "drinks"  would  be  placed  at  the  action  node, 
another  tree  structure  for  "John"  the  agent  node  and  so  on.  The  complexity  of  the  system  comes 
from  the  way  in  which  the  formulas,  considered  as  active  entities,  dictate  how  other  places  in  the 
template  should  be  filled. 


Thus,  the  "drink"  formula  above  can  be  thought  of  as  an  entity  at  a template  action  node,  seeking 
a liquid  object,  that  is  to  say  a formula  with  (FLOW  STLIFF)  as  its  right-most  branch,  to  put  at 
the  object  node  of  the  same  template.  This  seeking  is  preferential,  in  that  formulas  not  satisfying 
that  requirement  will  be  accepted,  but  only  if  nothing  satisfactory  can  be  found.  The  template 
finally  established  for  a fragment  of  text  is  the  one  in  which  the  most  formulas  have  their 
preferences  satisfied.  There  is  a general  principle  at  work  here,  that  the  right  interpretaion  "says 
the  least"  in  information-carrying  terms  This  very  simple  device  is  able  to  do  much  of  the  work 
of  a syntax  and  word-sense  ambiguity  resolving  program.  For  example,  if  che  sentence  had  been 
"John  drank  a whole  pitcher",  the  formula  for  the  "pucher  of  liquid”  would  have  been  preferred 
to  that  for  the  human,  since  the  subformula  (FLOW  STUFF)  could  be  appropriately  located 
within  it. 

A considerable  amount  of  squeezing  of  this  simple  canonical  form  of  template  is  necessary  to 
make  it  fit  the  complexity  of  language,  texts  have  to  be  fragmented  initially;  then,  in  fragments 
which  are,  say,  prepositional  phrases  there  is  a dummy  agent  imposed,  and  the  prepositional 
formula  functions  as  a pseudo-action.  There  are  special  "less  preferred”  orders  to  deal  with 
fragments  not  in  agent-action-object  order,  and  so  on. 

When  the  local  inferences  have  been  done  that  set  up  the  agent-action-object  templates  for 
fragments  of  input  text,  the  system  attempts  to  tie  these  templates  together  so  as  to  provide  an 
overall  initial  structure  for  the  input.  One  form  of  this  is  the  anaphora  tie,  of  the  sort  discussed 
for  the  monkeys  and  bananas  example  above,  but  the  more  general  form  is  the  case  tie. 
Assignment  of  these  would  result  in  the  template  for  the  last  clause  of  "He  ran  the  mile  in  a paper 
oag"  being  tied  to  the  action  node  of  the  template  for  the  first  clause  "He  ran  the  mile",  and  the 
tie  being  labelled  CONTainment.  These  case  ties  are  made  with  the  aid  of  another  class  of 
ordered  structures  called  paraplates,  that  are  attached  to  the  formulas  for  English  prepositions. 
So,  for  "ouiof"  there  would  be  at  least  six  ordered  parap'ates,  each  of  which  is  a string  of 
functions  that  seek  inside  templates  for  information  In  general,  paraplates  range  across  two,  not 
necessarily  contiguous,  templates  So,  in  analyzing  "He  put  the  number  he  thought  of  in  the 
table",  the  successfully  matching  paraplate  would  pin  down  the  dependence  of  the  template  for  the 
last  of  the  three  clauses  as  DlREction,  by  taking  as  argument  only  that  one  template  for  the  last 
clause  that  contained  the  formula  for  a numerical  table,  rather  than  a kitchen  table,  in  virtue,  in 
this  example,  of  the  function  in  that  paraplate  seeking  a similarity  of  head  (SIGN  in  this  case) 
between  the  two  object  formulas,  for  "number”  and  "table" 

The  structure  of  mutually  connected  templates  that  has  been  put  together  thus  far  constitutes  a 
"semantic  block",  and  if  it  can  be  constructed,  then,  as  far  as  the  system  is  concerned,  all  semantic 
and  referential  ambiguity  has  been  resoived  and  it  will  begin  to  generate  French  by  unwrapping 
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the  block  again.  The  gr  .eration  aspects  of  this  work  have  been  described  in  (Herskovits  1973). 
One  aspect  of  the  geiv.ral  notion  of  preference  is  that  the  system  should  never  construct  a deeper 
or  more  elaborate  serr  intic  representation  than  is  necessary  for  the  task  in  hand  and,  if  the  initial 
block  can  be  construe  ed  and  a generation  of  French  done,  no  "deepening”  of  the  representation 
will  be  attempted. 

However,  many  ex.  ’iples  cannot  be  resolved  by  the  methods  of  this  "basic  mode"and,  in 
particular  if  a word  sense  ambiguity,  or  pronoun  reference,  is  still  unresolved,  then  a unique 
semantic  block  of  templates  cannot  be  constructed  and  the  "extended  mode"  will  be  entered.  (5)  In 
this  mode,  p<*w  template-like  forms  are  extracted  from  exist.  ig  ones,  and  then  added  to  the 
template  pool  from  which  further  inferences  can  be  made.  So,  in  the  template  derived  earlier  for 
"john  drinks  wine"  the  system  enter.s  the  formula  for  "drinks”,  and  draws  inferences 
corresponding  to  each  case  sub-formula.  In  this  example  it  will  derive  template-like  forms 
equivalent  to,  in  ordinary  English,  "The  wine  is  in  John",  "The  wine  entered  John  via  an 
aperture"  and  so  on.  The  extracted  templates  express  information  already  implicitly  present  in  the 
text,  even  though  many  of  them  arc  partial  inferences:  ones  that  may  not  necessarily  be  true. 

Common-sense  in^*  .(ice  rules  .are  then  brought  down,  which  attempt,  by  a simple  strategy,  to 
construct  the  shortest  possible  chain  of  rule-linked  template  forms  from  one  containing  one  "of  its 
possible  referer  ts.  Such  a chain  then  constitutes  a solution  to  the  ambiguity  problem,  and  the 
preference  approach  assumco  that  the  shortest  chain  is  always  the  right  one.  So,  in  the  case  of 
"John  drank  the  wine  on  the  table  and  it  was  good",  the  correct  chain  to  "wine"  uses  the  two  rules 

I 1.  ((.^ANl  1)((SELF  lNXMOVECAUSE))(oREAL  2))-(l(:.JUDC)  2) 
or,  in  "semi-English", 

[animate- 1 cause-to-move-in-self  real-object-2]  -♦  [1  "'judges  2) 

12.  (1  BE  (GOOD  KIND))  « ((.'ANI  2)  WANT  1) 

or,  again, 

[1  IS  good]  « [animate-2  wants  1] 


These  ruies  are  only  partial,  that  is,  they  correspond  only  to  what  we  may  reasonably  look  out  for 
in  a given  situation,  not  to  what  MUST  happen.  The  hypothesis  here  is  that  understanding  can 
only  take  place  on  the  basis  of  simple  rules  that  are  contirmed  by  the  context  of  application.  In 
this  example  the  chain  constructed  may  be  expressed  as  [using  the  above  'square  bracket  notation' 
to  contain  not  a representation,  but  simply  an  indi.  ation,  in  English,  of  the  template  contents]: 


(5)  Footnote;  Wilks  1973b,  and  in  press. 
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[John  drank  the  wine) 

Template  1 

forwards 

[John  causes-to-move-in-sel  f 

wine) 

divide  Template  1 

inf. 

[John  mjudges 

w i ne) 

by  1 1. 

[John  wants 

wine) 

divide  line  above 

backwards  ‘ 
inf. 

twine  is 

good) 

by  I 2. 

[?it  is 

good) 

Template  3 

The  assumption  here  is  that  no  chain  using  other  inference  rules  could  have  reached  the  "table" 
solution  by  using  less  than  two  rules. 

The  chief  drawback  of  this  system  is  that  codings  consisting  entirely  of  primitives  have  a 
considerable  amount  of  both  vagueness  and  redundancy.  For  ex  Tiple,  no  reasonable  coding  in 
terms  of  structured  primitives  could  be  expected  to  distinguish,  .<  .y,  "hammer"  and  "mallet".  That 
may  not  matter  provided  the  codings  can  distinguish  impc  jntly  different  senses  of  words. 
Again,  a template  for  the  senience  "The  shepherd  tended  his  flock"  would  contain  considerable 
repetition,  each  node  of  the  template  trying,  as  it  were,  to  tell  the  whole  story  by  itscif.  Whether 
or  not  such  a system  can  remain  stable  with  a considerable  vocabulary,  of,  say,  several  thousand 
words,  has  yet  to  be  tested. 


It  will  be  evident  to  any  reader  that  the  last  two  systems  described,  Schank’s  and  my  own,  share  a 
great  deal  in  common.  Even  the  apparent  difference  in  notation  is  reduced  if  one  sees  the 
topological  similarity  that  results  from  considering  the  head  of  a formula  as  functioning  rather 
like  a Schank  basic  action.  If  one  thinks  of  the  dependencies  of  the  case  subparts  of  a formula, 
not  arranged  linearly  along  the  bottom  of  a tree,  but  radiating  out  from  the  head  in  the  center, 
then  the  two  diagrams  actually  have  identical  topologies  und^r  interpretation.  A difference  arises 
in  that  the  "filled-in  entity"  for  Schank  is  tf-“  conceptualisation  centered  on  the  basic  action, 
though  for  me  it  is  the  network  of  formulas  placed  in  relation  in  a template,  where  there  is  indeed 
a basic  action,  the  head  of  the  action  formula,  but  there  is  also  a basic  entity  in  the  agent  formula 
and  so  cn.  Or,  to  put  it  another  way,  both  what-is  and  what-is-expected  are  represented  in  the 
templates  the  agent  formula  represents  the  agent,  for  example,  but  the  left-hand  part  of  the 
action  formula  afso  represents  what  agent  wis  expected  or  sought,  as  in  the  (jANI  SUBJ)  sub- 
formula  of  the  "drink"  formula. 

Although  developed  in  isolation  initially,  these  two  systems  have  also  influenced  each  other  in 
more  r°cent  years,  probably  unconsciously.  For  example,  conceptual  dependency  now  emphasises 
the  agent-action-object  format  more  than  before,  and  is  less  "verb-centered"  and  timeless,  while, 
conversely,  my  own  system  now  makes  much  more  overt  use  of  rules  of  partial  information  than 
in  Its  eailier  versions.  Again,  both  systems  have  intellectual  connections  that  go  back  before  either 
generation  of  A I systems.  In  my  view,  both  these  systems  have  roots  in  the  better  parts  of  the 
Computational  Linguistics  movement  of  the  Fifties:  in  the  case  of  Schank’s  system,  one  may  think 
of  the  earlier  system.',  of  (Hays  1964)  and  (Lamb  1966),  and  the  arrow-structured  primitive  system 
of  (Farradene  1966)  In  the  case  of  my  own  system  there  are  clear  precedents  in  the  (Paiker- 
Rhodes  1961)  system  of  classification  and  the  early  semantic  structures  of  (Richens  1961)  and 


(Masteiman  1961).  In  1961  the  last  author  was  arguing  that  'what  is  needed  is  a discipline  that 
will  study  semantic  message  connection  in  a way  analogous  to  that  in  which  metamathematics  now 
studies  mathematical  connection,  and  to  that  in  which  mathematical  linguistics  now  studies 
syntactic  connection",  (ibid.,  p.3) 

This  historical  point  raises  a final  one  that  is.  I feel,  of  passing  interest.  There  seem  to  be  two 
research  styles  in  this  field;  one  is  what  might  be  called  the  "fully  finished  style",  in  which  the 
work  exists  onl,  in  one  complete  foim,  and  is  not  issued  in  early  or  developed  versions.  The  best 
example  of  this  is  Winograd’s  •.  ork.  The  other  type,  exemplified  by  all  the  other  authors 
discussed  here,  to  some  extent,  is  the  developing  style:  work  which  appears  in  a number  of 
versions  over  the  years,  one  hopes  with  gradual  improvements,  perhaps  in  attempts  to  tackle  a 
wider  range  of  linguistic  or  other  inferential  phenomena.  There  are  advantages  to  both  styles,  but 
even  in  the  Irtter  one  knows  that  any  proposed  structure  or  system  will,  in  the  end,  be  found 
wanting  in  the  balances  of  language,  so  it  can  only  be  a question  of  when  one  will  have  to 
abandon  it.  The  interesting  question,  and  one  to  which  no  answer  could  possibly  be  given  here,  is 
just  how  far  is  it  worth  pushing  . ay  given  structural  approach  before  starting  again  from  scratch? 
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6.  SOME  COMPARISONS  AND  CONTRASTS 


In  this  section  1 shall  compare  and  contrast,  under  some  eight  interconnected  headings,  the 
projects  described  in  the  body  of  the  paper,  This  is  not  easy  to  do,  particularly  when  the  present 
author  ..s  among  the  v'riters  discussed,  though  that  is  easily  remedied  by  the  reader’s  making  an 
appropriate  discount.  A more  serious  problem  is  that,  at  this  stage  of  research  in  artificial 
intelligence  and  natural  language,  the  most  attractive  distinch.iis  dissolve  on  more  detailed 
scrutiny,  largely  because  of  the  lack  of  any  precise  theoretical  statement  in  most  of,  if  not  all,  the 
major  projects.  There  are  those  who  think  that  it  therefore  follows  that  this  is  not  the  moment 
for  any  form  of  critical  comparison  in  this  field,  and  that  no  more  is  needed  than  a positive 
attitude  towards  all  possible  projects.  Only  those  who  fee!  that,  or.  the  contrary,  any  time  is  as 
good  as  any  other  for  the  discussion  of  intellectual  differences  in  the  hope  of  progress,  should 
read  on. 

It  must  be  admitted  right  away  that  the  selection  of  prcjects  discussed  above,  like  Wmograd's 
distinction  between  first  and  second  generation  systems,  on  which  the  selection  was  to  some  extent 
based,  cannot  be  defended  by  any  strict  defmitic.;'  one  that  would,  in  this  case,  include  all  the 
projects  described,  and  exclude  all  those  of  Wmograd’s  ’’first  generation".  One  might,  for 
example,  want  to  define  second  generation  systems  (in  the  study  of  natural  language  within  fhe  A I 
paradigm)  in  some  very  general  terms,  such  as  those  systems  which  (1)  contain  complex  semantic 
structures  for  the  representation  of  text  that  z.e  significantly  different  from  the  "surface  structure" 
of  the  input,  and  (2)  contain  cognate  structures  representing  conceptual  and  real  world  knowledge 
that  IS  not  explicitly  present  in  the  input  text.  Even  so  general  a description  of  a "frames"  type 
approach  would  not  cover  Charmak  or  Colby  with  the  first  point,  nor  Simmons  with  the  second, 
for  he  has  so  far  eschewed  all  concern  with  information  not  present  explicitly  in  the  input  text. 
Moreover,  the  second  point  would  certainly  cover  Winograd’s  own  work,  as  vrell  as  other  first 
generation  approaches,  so  it  is  clear  at  the  outset  of  any  comparison  that  there  is  not  even  a 
simple  and  unequivocal  definition  which  covers  all  and  only  the  projects  to  be  compared. 


Level  of  Representation 

One  important  line  of  current  dispute  among  the  second  generation  approaches  concerns  the 
appropriate  level  of  representation  for  natural  language.  On  the  one  hand  are  those  like  Colby, 
and  apparently  Charmak,  who  hold  that  the  repres»>ntation  of  language  can,  in  effect,  be  by 
means  of  itself  while,  on  the  other  hand,  there  are  those  like  Schank  and  myself  who  hold  that  the 
appropriate  level  of  computation  for  inferences  about  natural  language  is  in  some  reduced,  or 
primitive,  representation.  Simmons,  as  we  saw,  holds  an  intermediate  position.  I wrote  "appears" 
in  the  rase  of  Charmak  because  he  holds  that  his  structures  are  independent  of  any  particular 
level  of  representation,  or  rather,  that  they  could  be  realised  at  a number  of  levels  of 
representation,  depending  on  the  subject  area.  However,  there  is  no  doubt  that  the  representation 
m terms  of  predicates  that  he  offers  in  his  work  appears  to  be  in  one-to-one  correspondence  with 
English  words. 

The  strongest  low-level  approach  is  undoubtedly  that  of  Colby,  who  straightforwardly  faces  the 
enormous  mapping  problems  involved  if  the  structures  are  at  the  English  word  level.  It  is 
important  to  realise  that  this  dispute  is  ultimately  one  of  degree,  since  no  one  would  claim  that 
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even  location  recognised  by  an  inte.ligent  analyser  must  be  mapped  into  a "deep"  representation 
To  take  an  extreme  case,  any  system  that  mapped  Good  Morning  into  a deep  semantic 
representation  before  deciding  that  the  correct  response  was  also  "Good  Morning"  would  be 
making  a serious  theoretical  mistake. 

However  the  most  serious  argument  for  a non-superficial  representation  is  not  in  terms  of  the 
avoidance  of  mapping  difficulties,  but  in  terms  of  theoretical  perspicuity  of  the  primitive 
structures  Th.s  argument  is  closely  tied  to  the  defense  of  semantic  primitives  in  general,  which  is 
a large  subject  not  to  be  undertaken  here.  One  of  the  troubles  about  semantic  primitives  is  that 
they  are  open  to  bad  defenses,  which  decrease  rather  than  increase  their  plausibility.  For  example, 
some  users  of  them  for  linguistic  representation  have  declared  them  to  have  some  sort  of  objective 
existence  rnd  have  implied  that  there  is  a "right  set"  of  primitives  open  to  empirical  discovery 
On  that  view  the  essentially  linguistic  character  of  structures  of  primitives  is  lost,  for  they  then 
might  as  well  be  strings  of  binary  numbers,  or  something  equally  opaque  and  non-linguistic.  No 
ere^at  deal  of  thought  is  required  to  see  that  that  simply  could  not  be  the  case.  What  is  the  case  is 
That  there  is  a considerable  amount  of  psychological  evidence  that  people  are  able  to  recall  the 
content  of  what  they  hear  and  ur.dcrsland  without  being  able  to  recall  either  the  actual  words  or 
the  syntactic  structure  used  There  is  large  literature  on  this  subject,  from  which  two  sample 
refeiences  would  be  (Wettler  1973)  and  (Johnson-Laird  1974). 

These  results  are,  of  course,  no  proof  of  the  existence  of  semantic  primitives,  but  they  are 
undoubtedly  supporting  evidence  of  the.’  plausibility,  as  is,  on  a different  plane,  the  result  from 
the  encoding  of  the  whole  Webster’s  1 hird  International  Dictionary  at  Systems  Development 
Corporation  where  it  was  found  that  a rank-ordered  frequency  count  of  the  words  used  to  define 
other  words'in  that  vast  dictionary  was  a list  (omitting  "the"  and  "a")  which  corresponded  almost 
item-for-item  to  a plausible  list  of  semantic  primitives,  derived  a priori,  by  those  actually 
concerned  to  code  the  structure  of  word  and  sentence  meanings. 

It  IS  important  to  distinguish  the  dispute  about  level  from  tne  closely  connected  topic  that  I shall 
call  the  centrality  of  the  knowledge  required  by  a language  understanding  system. 


Centrality 

What  1 am  calling  the  centrality  of  certain  kinds  of  information  concerns  not  its  level  of 
representation  but  its  non-specificity.  again  a contrast  can  be  drawn  between  the  sorts  of 
information  required  by  Charniak's  system,  on  the  one  hand,  and  that  required  by  Schanks  and 
my  own  on  the  other.  Charniak’s  examples  suggest  that  the  fundamental  form  of  information  is 
highly  specific  (6)  to  particular  situations,  like  parties  and  the  giving  of  presents,  while  the  sorts  of 
information  central  to  Schank’s  and  my  own  systems  are  general  partial  assertions  about  human 
wants,  expectations,  and  so  on,  many  of  which  are  so  genera!  as  to  be  almost  vacuous  which,  one 
might  argue,  is  why  their  role  in  understanding  has  been  ignored  for  so  long 

’ 1 were  a reasonably  fluent  speaker  of.  say,  German,  1 might  well  not  understand  a German 
conversation  about  birthday  presents  unless  1 had  detailed  factual  information  about  how 
Germans  organise  the  giving  of  presents,  which  might  be  considerably  different  from  the  way  we 
do  It  Conversely,  of  course,  1 might  understand  much  of  a technical  article  about  a subject  in 


(6)  In  Charniak’s  most  recent  paper  (1974),  he  gives  much  more  general  rules,  such  as  his  "rule  of 
significant  sub-action",  mentioned  earlier. 
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which  1 was  an  expert,  even  though  I knew  very  little  of  the  language  in  which  it  was  written. 
These  are  certainly  considerations  that  tell  for  Charniak’s  approach,  and  it  is  perhaps  a paradox 
that  the  sort  of  natural  language  understander  that  would  tend  to  confirm  his  assumptions  would 
be  one  concerned  with  discourse  about,  say,  the  details  of  repairing  a motor  car,  where  factual 
information  is  what  is  central,  yet,  ironically,  Charuiak  has  concentrated  on  something  as  general 
as  children’s  stories,  with  their  need  of  deep  assumptions  about  human  desires  and  behavior. 

In  the  end  this  difference  may  again  turn  out  to  be  one  of  emphasis,  and  of  what  is  most 
appropriate  to  different  subject  areas,  though  there  may  be  a very  general  i.ssue  lurking 
somewhere  here.  It  seems  to  me  not  a foolish  question  to  ask  whether  much  of  what  appears  to  be 
about  naiutal  languagf  in  A.I.  research  is  in  fact  about  language  at  all.  Even  if  it  is  not  that  may 
in  no  way  detract  from  its  value.  Newell  (Moore,  Newell  1973)  has  argued  that  A.I.  work  i.s  in 
fact  "theoretical  psychology",  in  which  case  it  could  ha’^dly  be  research  on  natural  language. 
When  describing  Winograd’s  work  earlier  in  the  paper,  I raised  this  question  in  a weak  form  by 
asking  whether  his  definition  of  "pickup"  had  anything  to  do  with  the  natural  language  use  of  the 
word,  or  whether  it  was  rather  a description  of  how  his  system  picked  something  up,  a quite 
different  matter. 

Suppose  we  generalise  this  query  somewhat,  by  asking  the  apparently  absurd  question  of  what 
would  be  wrong  with  calling,  say,  Charniak’s  work  an  essay  on  the  Socio-Economic  Behavior  of 
American  Children  Under  Si.ess?  In  the  case  of  Charniak’s  work  this  is  a facetious  question, 
asked  only  in  order  to  make  a point,  but  with  an  increasing  number  of  systems  in  A.I.  being 
designed  not  essentially  to  do  research  on  natural  language,  but  in  order  to  have  a natural 
language  "front  end”  to  a system  that  is  essentially  intended  to  predict  chemical  spectra,  or  play 
snakes  and  ladders  or  whatever,  the  question  becomes  a serious  one  It  seems  to  me  a good  time 
to  ask  whether  we  should  expect  advance  in  understanding  natural  language  from  those  tackling 
the  problems  head  on,  or  those  concerned  to  build  a "front  end"  It  is  clearly  the  case  that  cnv 
piece  of  knowledge  whatever  could  be  essential  to  the  understanding  of  some  story.  The  question 
IS,  does  it  follow  that  the  specification,  organisation  and  formalisation  of  that  knowledge  is  the 
study  of  language,  because  if  it  is  then  all  human  enquiry  from  physics  and  history  to  medicine  is 
a linguistic  enterprise.  And,  of  course,  that  possibility  has  actually  been  entertained  within  certain 
strains  of  modern  philosophy 

However,  I am  not  trying  here  to  breathe  fresh  life  into  a philosophical  distinction,  between  being 
about  language  and  not  being  about  language,  but  rather  introducing  a practical  distinction, 
(which  IS  also  a consideration  in  favour  of  opting,  as  I have,  to  work  on  very  general  and  central 
areas  of  knowledge)  between  specific  knowledge,  and  central  knowledge  without  which  a system 
could  not  be  said  to  understand  the  language  at  all.  For  example,  I n ight  know  nothing  of  the 
arrangement  of  American  birthday  parties,  but  could  not  be  accused  of  not  understanding  English 
even  though  I failed  to  understand  some  particular  children’s  story  Yet,  if  I did  not  have 
available  some  very  general  partial  inference  such  as  the  one  about  people  being  hurt  and  falling, 
or  one  about  people  endeavouring  to  possess  things  that  they  want,  then  it  is  quite  possible  that 
my  lack  of  understanding  of  quite  simple  sentences  would  cause  observers  to  think  that  I did  not 
understand  English.  An  interesting  and  difficult  question  that  then  arises  is  whether  those  who 
concentrate  on  central  areas  of  discourse  could,  in  principle,  weld  their  bodies  of  inferences 
together  in  such  a way  as  to  create  a wider  system:  whether,  to  put  the  matter  another  way, 
natural  language  Is  a whole  that  can  be  built  up  from  parts? 
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Pheiionieiiological  Level 

Another  distinction  that  can  be  confused  with  the  central-specific  one  is  that  of  the 
"phenomenological  levels"  of  inference  in  an  iind"istanding  system.  I mean  nothing  daunting  by 
the  phrase;  consider  the  action  eating  which  is.  as  a matter  of  anatomical  fact,  quite  often  an  act 
of  bunging  the  bones  of  my  ulna  and  radius  (in  my  arm)  close  to  that  of  my  lower  mandible  (my 
jaw).  Yet  clearly,  any  system  of  common  sense  inferences  that  considered  such  a truth  when 
reasoning  about  eating  would  be  making  a mistake.  One  might  say  that  the  phenomenological 
level  of  the  analysis  was  wrong  even  though  all  the  inferences  it  made  were  true  ones.  The  same 
would  be  true  of  any  A. I.  system  that  made  everyday  inferences  about  physical  objects  by 
considering  their  ’antum  structure. 

Schank’s  analysis  of  eating  contains  the  information  that  is  done  by  moving  the  hands  to  the 
mouth,  and  it  might  be  argued  that  even  this  is  going  too  far  from  the  "meaning"  of  eating, 
whatever  that  may  be,  towards  geneially  true  information  about  the  act  which,  if  always  inferred 
about  all  acts  of  eating,  will  carry  the  system  unmanageably  far. 

There  is  no  denying  that  this  sort  of  information  might  be  useful  to  have  around  somewhere, 
that,  in  Minsky's  terms,  the  "default"  value  of  the  instrument  for  eating  is  the  hand  brought  to  the 
mouth,  so  that,  if  we  have  no  contrary  'nformation,  then  that  is  the  way  to  assume  that  any  given 
act  of  eating  was  performed.  Nonetheless,  there  clearly  is  a danger,  and  that  is  all  I am  drawing 
attention  to  here,  of  taking  inferences  to  a phenomenological  level  beyond  that  of  common  sense. 
A clearer  case,  in  my  view,  would  be  Schank’s  analyses  (1974a)  of  mental  activity  in  which  all 
actions,  such  as  kicking  a ball,  say.  are  preceded  by  a mental  action  of  conceiving  or  deciding  to 
kick  a ball.  This  is  clearly  a level  of  analysis  untrue  to  common  sense,  and  which  can  have  only 
harmful  effects  in  a system  intended  to  mimic  common  sense  reasoning  and  understanding. 


Decoupling 

Another  general  issue  in  dispute  concerns  what  I shall  call  liecoupling,  which  is  whether  or  not  the 
aclual  parsing  of  te.xt  or  dialogue  into  an  "understanding  system"  is  essential.  Charniak  and 
Mmsky  believe  that  this  initial  "parsing"  can  be  effectively  decoupled  from  the  interesting 
inferential  work  and  simply  assumed  But,  in  my  view,  that  is  not  so,  because  many  of  the  later 
inferences  would  actually  have  to  be  done  already,  in  order  to  have  achieved  the  initial  parsing, 
and  so  the  assumption  of  decoupling  can  lead  to  something  like  a circularity.  For  example,  in 
analysing  "He  shot  her  with  a colt",  we  cannot  ascribe  any  structure  at  all  until  we  can  make  the 
inference  that  guns  rather  than  horses  are  instruments  for  shooting 

The  inferences  require"'  to  resolve  word  sense  ambiguities,  and  those  required  to  resolve  pronoun 
relrrence  problems,  are  not  of  different  types;  often  the  two  problems  occur  in  a single  sentence 
and  must  be  resolved  together.  But  Charniak's  decoupling  has  the  effect  of  completely  separating 
these  two  closely  related  linguistic  phenomena  in  what  seems  to  me  an  unrealistic  manner.  His 
system  does  inferencing  to  resolve  pronoun  ambiguiiies,  while  sense  ambiguity  is  presumably  to  be 
done  in  the  future  by  some  other,  ultimately  recoupled,  system 

Anotlie*^  way  of  pointing  up  the  difference  between  the  attitudes  of  second  generation  systems  to 
decoupling,  in  relatio.a  to  the  first  generation,  is  by  describing  the  role  of  syntax  analysis  in  them. 
A^  we  .saw,  syntax  was  the  heart  of  Winograd’s  system,  but  both  levels  of  frame  approach 
discount  syntax  analysis,  though  for  very  different  reasons:  Charniak  does  so  because  it  is  part  of 
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the  initial  parsing  from  which  his  inferential  work  has  been  decoupled.  Schank  and  I do  so 
because  we  believe  semantic  analysis  to  be  fundamental,  and  that  in  an  actual  implementation  the 
results  of  syntactic  analysis  can  all  be  achieved  by  a sufficiently  powerful  semantic  analyser.  And 
this  last  assumption  is  confirmed  by  the  limited  degree  of  success  that  the  two  semantic  analysers 
have  actually  achieved  in  operation. 


Availability  of  surface  structure 

An  issue  close  to  that  of  the  appropriate  level  of  representation  in  a system  is  that  of  the 
availability  of  the  surface  structure  of  the  language  analysed;  or,  to  put  it  more  crudely,  the 
availability  during  subsequent  analysis  of  the  actual  words  being  analysed.  These  are  clearly 
available  in  Colby's,  and  are  indirectly  available  in  Simmons’s,  Winograd’s  and  my  own  systems, 
but  Schank  makes  a point  of  the  importance  of  their  non-availability,  on  the  grounds  that  an 
ideal  representation  should  be  totally  independent  of  the  input  surface  structure  and  words. 
There  are  both  theoretical  and  practical  aspects  to  this  claim  of  Schank’s:  in  the  limit,  the  order 
of  the  sentences  of  a text  is  part  of  its  surface  structure,  and  presumably  it  is  not  intended  to 
abandon  this  "superficial  information".  In  one  of  his  recent  papers  (1974b)  Schank  seems  to  have 
accepted  some  limitation  on  the  abandonment  of  surface  structure. 

The  other,  practical,  point  concerns  the  form  of  representation  employed:  in  the  (197?) 

implementation  of  Schank’s  system  using  an  analyser  of  input  text,  a memory  and  a generator  of 
responses,  it  was  intended  that  nothing  should  be  '.  ansferred  from  the  input  program  to  the 
output  program  except  a representation  coded  in  the  structures  of  primitives  discussed  earlier.  (7) 
The  question  that  arises  is,  can  that  structure  specify  and  distinguish  word-senses  adequately 
without  transferring  information  specifically  associated  with  the  input  word?  Schank  clearly 
believes  the  answer  to  this  question  is  yes,  but  that  cannot  be  considered  established  by  the  scale 
of  computations  yet  described  in  print. 

A suitable  environment  in  which  to  consider  the  question  is  that  of  translation  from  one  language 
(O  another,  suppose  we  are  analysing  a sentence  containing  the  word  "nail",  meaning  a physical 
object.  It  is  clear  that  the  translation  of  that  word  into  French  should  not  be  the  same  as  the 
translation  for  ".screw"  or  "peg".  Yet  is  it  plausible  that  any  description  of  the  function  of  these 
three  entities  entirdy  in  terms  of  semantic  primitives,  and  without  any  explicit  mention  of  the  word 
name  and  its  connection  to  its  French  equivalent,  will  be  sufficient  to  ensure  that  only  the  right 
match  is  made? 


Application 

This  point  is  a generalisation  of  the  last  two,  and  ccnccins  the  way  in  which  different  systems 
display,  in  the  structures  they  manipulate,  the  actual  procedures  of  application  of  those  structures 
to  input  text  or  dialogue.  This  is  a matter  different  from  boiri  'hit  of  the  availability  of  the 


(7)  This  point  This  point  is  to  some  extent  h)’poth'*;c?l  sine**,  as  we  saw,  Schank’s 
conceptualisations  still  do  contain,  or  appear  to  contain,  mat  fface  items;  in  particular  nouns, 

adjectives  and  adverbs.  However,  this  is  a transitiona'  mai.  ;nd  they  are  in  the  course  of 

rejilacement,  as  noted,  by  non-superficial  items. 
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surface  structure,  and  of  a computer  implmenlation  of  the  system  In  the  case  of  Colby's  patterns, 
for  example,  the  form  of  their  application  to  the  input  English  is  clear,  even  though  the  matching 
involved  could  be  achieved  by  many  different  implementation  algorithms.  In  the  case  of  my  own 
system,  1 hold  the  same  to  be  true  of  the  template  structures,  even  though  by  the  time  the  input 
has  reached  the  canonical  template  form  it  is  considerably  different  from  the  input  surface 
structiiie  The  system  at  the  extreme  end  of  any  scale  of  perspicuity  of  application  is  Winograd's, 
where  the  procedural  notation,  by  its  nature,  tends  to  make  clear  the  way  in  which  the  structures 
aie  applied.  At  the  other  end  are  the  systems  of  Schank  and  Charniak,  where  no  application  is 
specified,  which  means  that  the  representations  are  not  only  compatible  with  many  implementation 
algorithms,  which  does  not  matter,  but  are  also  compatible  with  many  systems  of  linguistic  rules, 
whose  specification  is  an  essential  piece  of  inquiry,  and  whose  subsequent  production  may  cause 
the  basic  system  to  be  fundamentally  different. 

English  prepositions  will  serve  as  an  example:  in  Schank's  case  notation  there  is  no  indication  of 
how  the  case  discriminations  are  actually  to  be  applied  to  English  prepositions  in  text.  So,  for 
example,  the  preposition  "in"  can  correspond  to  the  containment  case,  time  location,  and  spatial 
location,  among  others.  As  we  saw  earlier,  the  discrimination  involved  in  actual  analysis  i.s  a 
matter  of  specifying  very  delicate  semantic  rules  ranging  over  the  basic  semantic  structures 
employed.  Indeed,  the  structures  and  case  system  themselves  seem  to  me  to  be  essentially 
dependent  on  the  natuie  and  applicability  of  such  rules.  (8)  and  so  this  application  of  toe  system 
should  have  an  obvious  place  in  the  overall  structures.  It  is  not  something  to  be  delegated  to  a 
mere  "implementation".  If  enough  of  the  linguistic  intractables  (9)  of  English  analysis  were  to  be 
delegated  out  of  the  representation,  A.l.  would  be  offering  no  more  to  the  analysis  of  natural 
language  than  the  logicians  who  proffer  the  predicate  calculus  as  a plausible  structure  for  English. 

In  some  of  his  more  recent  writings  VVinograd  has  begun  to  develop  a view  that  is  considerably 
stronger  than  this  "application"  one;  in  his  view  the  control  structure  of  an  understanding  program 
IS  Itself  of  theoretical  significance,  for  only  in  that  way,  he  believe.s,  can  natural  language 
programs  of  great  sue  and  complexity  remain  perspicuous. 


Forward  inference 

Another  outstanding  dispute  concerns  whether  one  should  make  massive  forward  inferences  as 
one  goes  through  a text,  keeping  all  one's  expectations  intact,  as  Charniak  and  Schank  hold,  or 
whether,  as  1 hold,  one  should  adopt  some  "laziness  hypothesis"  about  understanding,  and 
generate  deeper  inferences  only  when  the  system  is  unable  to  solve,  say,  a referential  problem  by 
more  superficial  methods. 


(8)  This  IS  not  meant  to  be  just  bland  asseition  I have  written  at  some  length  on  the  relations 
between  application  and  the  theoretical  status  of  linguistic  theories  in  (Wilks  1974). 

(9)  The  differences  between  Minsky's  (1974)  notion  of  "default  value"  and  what  I have  called 
"preference”  can  be  pointed  up  in  terms  of  application  Min.sky  suggests  "gun"  as  the  default 
value  of  the  instrument  of  the  action  of  shooting,  but  I would  claim  that,  in  an  example  like  the 
earlier  "He  shot  her  with  a colt", we  need  to  be  able  to  see  in  the  structure  assigned  whether  or  not 
what  IS  offered  as  the  apparent  instrument  is  in  fact  an  instrument,  and  whether  it  is  the  default 
or  not  In  other  words,  we  need  sufficient  structure  of  application  to  see  not  only  that  "shooting" 
piefers  an  instniment  that  is  gun,  but  also  why  it  will  choose  the  sense  of  "colt"  that  is  a gun 
rather  than  the  one  which  is  a horse. 
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Altlioiigli  Schank  sometimes  writes  of  a system  making  "all  possible"  inferences  as  it  proceeds 
through  a text,  this  is  not  in  fact  the  heart  of  the  dispute,  since  no  one  would  want  to  defend  any 
stiong  definition  of  the  term  "all  possible  inferences"  Charniak’s  argument  is  that,  unless  certain 
forward  inferences  are  made  during  an  analysis  of,  say,  a story  -forward  inferences,  that  is,  that 
are  not  problem-diiven,  not  made  in  response  to  any  particular  problem  of  analysis  t/ien  known  to 
the  system— then,  as  a matter  of  empirical  fact,  the  system  will  not  in  general  be  able  to  solve 
ambiguity  or  reference  problems  that  arise  later,  because  it  will  never  in  fact  be  possible  to  locate 
(while  looking  backwards  at  the  text,  as  it  were)  the  points  where  those  forward  inferences  ought 
to  have  been  made.  This  is.  in  very  crude  siimrr.ary,  Charniak's  case  against  a purely  problem- 
driven  inferencer  in  a natural  language  understander. 

A difficulty  with  this  argument  is  the  location  of  an  example  of  text  that  confirms  the  point  in  a 
non-contentious  manner.  Charniak  has  found  an  excerpt  from  a book  describing  the  life  of  apes 
in  which  It  IS  indeed  hard  to  locate  the  reference  of  a particular  pronoun  in  a given  passage. 
Charniak’s  case  is  that  it  is  only  possible  to  do  so  if  one  has  made  certain  (non-problem 
occasioned)  inferences  earlier  in  the  story.  But  a number  of  readers  find  it  quite  hard  to  refer 
that  particular  pronoun  anyway,  which  might  suggest  that  the  text  was  simply  badly  written. 

This  IS  a difficult  matter  about  which  to  be  precise:  it  would  be  possible,  for  example,  to  agree 
with  Charniak's  argument  and  still  construct  a purely  problem-driven  inferencer  on  the  ground 
that,  at  the  moment,  this  is  the  only  way  one  can  cope  with  the  vast  majority  of  inferences  for 
understanding,  since  any  system  of  inferences  made  in  response  to  no  particular  problem  in  the  text 
IS  too  haid  to  control  in  practice  Indeed,  it  is  noticeable  that  the  most  recent  papers  of  Schank 
(197-fa  and  197-fb)  and  Charniak  (1974)  have  been  considerably  less  forward-inference-oriented 
than  earlier  ones.  (10) 

This  dispute  is  perhaps  only  one  of  degree,  and  about  the  possibil.ty  of  defining  a degree  of 
forward  infcretice  that  aids  the  solution  of  later  semantic  problems  without  going  into  unnecessary 
depth.  (It)  This  might  be  an  area  where  psychological  investigations  would  be  of  enormous 
help  to  the  workers  in  A. I. 


The  justification  of  systems 

Finally,  one  might  usefully,  though  briefly,  contrast  the  different  modes  of  justification  implicitly 
appealed  to  by  the  systems  described  earlier  in  this  paper.  These  seem  to  me  to  reduce  to  four: 

(i)  in  terms  of  the  power  of  the  inferential  system  employed.  This  form  of  justification  has 
underlain  the  early  predicate  calculus-based  language  programs,  and  is  behind  Hayes' 
(1974)  recent  demand  that  any  formalism  for  natural  language  analysis  should  admit 
of  a set  theoretical  semantics,  in  the  Tarskian  sense,  so  as  tc  gain  "intellectual 
respectability",  as  he  puts  it.  The  same  general  type  of  justification  is  appealed  to  in 
some  degree  by  systems  with  PLANNER -type  formalisms; 


(|Ci)  A particularly  interesting  withdrawal  of  a strong  forwaid  inference  thesis  is  hidden  aw.ay  on 
p 28?  of  (Riegr>r  1974),  but  has  been  located  by  the  keen  eye  of  E.  Charniak. 

(II)  This  may  be  no  more  than  a psychological  restatement  of  what  used  to  be  calleo  (Hayes  I97l) 
(Saiidewall  1972)  the  "frame  problem"  (no  relation.  P E ). 
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(li)  in  terms  of  the  provision  and  formalisation,  in  any  terms  including  English,  of  the  'Oits  of 
knowledge  required  to  understand  areas  of  discourse; 

(ill)  in  terms  of  the  actual  performance  of  a system,  implemented  on  a computer,  at  a task 
agreed  to  demonstrate  understanding; 

(iv)  in  terms  of  the  linguistic  and/or  psychological  plausibility  of  the  proffered  system  of 
representation. 

Oversimplifying  considerably,  one  might  say  that  Charniak’s  system  appeals  mostly  to  (ii)  and 
somewhat  to  (i)  and  (iv);  Winograd’s  to  (in)  and  somewhat  to  the  other  three  categories;  Colby’s 
(as  regards  its  natural  language,  rather  than  psychiatric,  aspects)  appeals  almost  entirely  to  (iii); 
Simmons  largely  to  (iv).  and  Schank’s  and  my  own  to  differing  mixtures  of  (ii),  (iii)  and  (iv). 

In  the  end,  of  course,  only  (in)  counts  for  empiricists,  but  there  is  considerable  difficulty  in  getting 
all  parties  to  agree  to  the  terms  of  a test.  (12)  A cynic  might  say  that,  in  the  end,  all  these  systems 
analyse  the  sentences  that  they  .analyse  or,  to  put  the  same  point  a little  more  cheoreticaliy,  there  is 
a sense  in  which  systems,  those  descrioed  here  and  those  elsewhere,  each  define  a natural 
language,  namely  the  o.ie  to  which  it  applies.  The  difficult  question  is  the  extent  to  which  those 
many  and  small  natural  languages  resemble  English. 


Conclusion 

The  last  section  'tressed  areas  of  current  disagreement,  but  there  would,  if  votes  were  taken,  be 
considerable  agreement  among  A. I workers  on  natural  language  about  where  the  large  problems 
of  the  immediate  future  are;  the  need  for  a good  memory  model  has  been  stressed  by  Schank 
(1074a),  and  many  would  add  the  need  for  an  extended  procedural  theory  of  texts,  rather  than  of 
individual  example  sentences,  and  for  a more  sophisticated  theory  of  reasons,  causes,  and  motives 
for  use  in  a theory  of  t iderstandmg  Many  might  also  be  persuaded  to  agree  on  the  need  to  steer 
between  the  Scylla  of  trivial  first  generation  iniplementations  and  the  Charybdis  of  utterly 
fantastic  ones  By  the  latter.  1 mean  projects  that  have  been  seriously  discussed  but  never 
implemented  for  obvious  reasons,  that  would,  say,  enable  a dialogue  program  to  discuss  whether 
or  not  a participant  in  a given  story  "felt  guilty",  and  if  so  why. 

The  last  disease  has  sometimes  had  as  a major  symptom  an  extensive  use  of  the  word 
"pragmatics"  (though  this  can  also  indicate  quite  benign  conditions  in  other  cases),  along  with  the 
implicit  claim  that  "semantics  has  been  solved,  so  we  should  get  on  with  the  pragmatics".  It  still 
nee  is  repeating  that  there  is  no  sense  whatever  m which  the  .semantics  of  natural  language  has 
been  solved.  It  is  still  the  enormous  barrier  it  has  always  been,  even  if  a few  dents  in  its  surface 
are  beginning  to  appear  here  and  there.  There  are  still  great  difficulties  both  systematic  and 
linguistic,  even  if  we  stick  to  the  simplest  examples  that  present  no  difficulty  to  the  human  reader- 
-and  It  must  be  admitted  that  it  has  been  one  of  the  persistent  faults  of  the  A.I.  paradigm  of 
language  that  it  h.as  spent  too  much  time  on  puzzles  examples 


(12)  Though  .an  interesting,  and  potentially  revolutionary,  distinction  seems  to  have  been 
intioduced  by  a recent  reviewer  of  manv  of  the  sy.stems  discussed  here,  between  the  functioning 
of  a program  and  a "program  in  itself”:  "Only  Winograd  describes  a program  that  is  sufficiently 
impressive  in  itself  to  force  us  to  take  his  ideas  seriously.  The  techniques  of  the  others  have  to 
get  by  on  whatever  intuitive  appeal  they  can  muster",  (hard  1974) 
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An  example  ot  tlie  foimer  would  be  the  development  of  a dynamic  system  of  understanding-  texts 
or  stones  that  had  any  capacity  to  recovoi  after  having  its  expectations  satisfied  and  then, 
subsequently,  frustrated.  At  present  no  system  of  the  sort  desciibed,  whether  of  demons, 
preference  or  whatever,  has  any  such  capacity  to  recover.  The  situation  is  quite  different  from 
that  in  a dialogue,  as  in  Winograd's  system,  where,  on  being  given  each  new  piece  of  information, 
the  system  checks  it  against  what  it  knows,  to  see  if  it  is  being  contr.adicted,  and  then  behaves  in 
an  appropriately  puzzled  way  if  it  is.  In  frame  or  "expectation”  systems  it  is  all  too  easy  to 
construct  apparently  trick,  but  basically  plausible,  examples  that  satisfy  what  was  being  looked  for 
and  then  overtut n it.  That  possibility  is  already  built  into  the  notion  of  frame  or  expectation. 
An  example  of  Phil  Hayes’  against  my  own  system  will  serve:  consider  "The  hunter  licked  his 
gun  all  over,  and  the  stock  tasted  especially  good’-.  What  is  meant  by  "stock”  is  clearly  the  stock 
piece  of  the  gun,  but  any  preference  system  like  mine  that  considers  the  two  senses  of  "stock",  and 
sees  that  an  edible,  soup  sense  of  "stock"  is  the  preferred  object  of  the  action  "taste",  will  infallibly 
opt  f,,  the  wrong  sense.  Any  frame  or  expectation  system  is  prone  to  the  same  general  kind  of 
counter-example. 

In  particular  cases  like  this  it  is  easy  to  suggest  what  might  be  done:  here  we  might  suggest  a 
preference  attached  to  the  formula  for  anything  that  was  essentially  part  of  another  thing  (stock  = 
"part  of  gun"  in  this  case),  so  that  a local  search  was  made  whenever  the  "part  of"  entity  was 
mentioned,  and  the  satisfaction  of  that  search  would  always  be  'he  overriding  preference.  But 
that  IS  not  the  same  as  a general  solution  to  the  problem,  which  used  to  be  called  that  of  "topic"  in 
the  computational  semantics  of  the  Fifties.  There  are  no  solutions  to  this  problem  available  here 
and  now.  though  some  suggestions  have  been  made  by  Abelson  (1974)  and  McDermott  (197-i  . 

A closely  related,  but  equally  intractable,  problem  is  that  of  how  to  combine  highly  specific  factual 
information  within  a general  semantic  structuring.  Systems  like  Charniak’s  are,  as  we  saw, 
concerned  with  specific  rather  than  conceptual  information,  but  there  are  quite  simple  "semantic 
specificity"  problems  that  one  could  not  reasonably  expect  to  be  tackled  even  in  a system  devoted 
to  the  handling  of  facts,  as  can  be  seen  by  contrasting  the  sentences: 

The  deer  came  out  of  the  wood. 

The  grub  came  out  of  the  wood. 


where  we  might  safely  assume  that  readers  would  assign  quite  different  senses  to  "wood"  in  the 
two  cases  simply  on  the  basis  of  the  two  different  agents.  No-one.  to  my  knowledge,  has  suggested 
any  general  method  for  tackling  such  elementary  examples. 

But.  to  finish  on  the  bright  side,  it  is  important  to  stress  thai  there  is  indeed  an  A.I.  paradigm  ot 
language  understanding  in  existence.(13)  one  that  embraces  first  and  second  generation 
approaches,  and  w.iich  goes  back,  I suggested,  to  a considerable  amount  of  earlier  work  in 
computational  linguistics.  It  can  be  distinguished  by  a catalogue  of  neglect  by  conventional 
linguistics  that  can  be  summarised  under  three  heads. 

(i)  theories  of  language  must  have  procedural  applicatwn  to  the  subject  matter  that  could  in 
principle  result  in  computer  application  and  subsequent  empirical  test; 

(n)  theories  of  language  must  deal  with  it  in  a communicative  context,  one  amenable  to 
empirical  assessment  Merely  sorting,  as  generative  theories  were  designed  to  do,  is 
not  enough; 

One  of  the  very  few  acknowledgements  of  this  fact,  of  the  possibility  of  an  A.I  paradigm  of 
language,  from  a linguist  is  (Fillmore  1974) 


(iii)  theories  of  language  must  also  be,  in  a clear  sense,  theories  of  the  formalisation  and 
organisation  of  knowledge.  If  they  are  not  then  we  can  know  in  advance  that  they 
can  never  tackle  the  problem  of  language  understanding. 
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